The hottest Data Platforms Substack posts right now

Big cloud vendors will keep rebranding and repositioning their data products to appear 'AI-first', adding marketing noise and confusion about which tools to use.
Almost all companies still rely on Excel, SFTP, and manual exports. Only a small share chase flashy AI while most need simple tools to convert spreadsheets into reliable data pipelines.
The modern data stack will be shaken by acquisitions, price changes, and fragile pipelines, forcing many teams to rebuild infrastructure and turn AI proofs-of-concept into production-ready foundations.

Becoming a data analyst or engineer through bootcamps is becoming less prevalent due to economic factors.
Analytics leaders face challenges in setting boundaries and avoiding overlap with finance teams in accounting functions.
Decentralized data team setups are generally more efficient, and the future may see more of this with changes in tax regulations.

Systems of record aren’t going away—businesses still need a single, reliable source of truth, which will increasingly live across warehouses, lakehouses, and operational systems paired with semantic layers and control planes.
AI agents span many systems and act on data, so they need explicit metric definitions, precedence rules, and conflict-resolution encoded where the truth lives, not left to human judgment.
Operational apps will shift into programmatic state machines with APIs, and the winners will be the products that provide durable truth, governance, and safe agent orchestration rather than just new UIs.

Data platforms are the next step after the Modern Data Stack, offering enhanced productivity, rapid iteration, and cost efficiency.
The evolution of technology is not linear, but branches out in many directions, leading to multiple 'next' possibilities.
New data platforms focus on integration, flexibility, and control, providing solutions for core issues like missing design, data quality, and integration challenges.

Get a weekly roundup of the best Substack posts, by hacker news affinity:

The concept of a Customer Data Platform (CDP) is evolving with a focus on defining its functionality more clearly.
There is a trend towards composable CDP solutions, allowing for flexibility but also potential complexity.
The key value of a CDP lies in activation - using customer data to create targeted audiences for more efficient marketing strategies.

The annual cybersecurity directory will stop after 2025 because publishing a complete, up-to-date vendor list risks enabling competitors to copy the database. This protects the business value of the dataset.
AI Security is exploding — about 290 companies founded 2022–2025 use AI to secure systems or apply AI to security tasks. That rapid growth means many startups will be acquired and the category will need frequent updates.
A forthcoming book will comprehensively profile all AI security companies using the full dataset, providing the first market-wide view of the space. It will be published in mid-January with signed copies available at the RSA conference.

Having modern tools doesn't guarantee providing value - it's more about how analytics teams use the tools to drive organizational change.
The focus should be on delivering value to the organization rather than just building data platforms or using the most modern tools.
Start simple with the minimum viable data stack and only add complexity when necessary - focus on solving real problems and evaluating tools based on problem-solving, maintenance, and scalability.

Operational use cases with Kafka and Flink are crucial for business operations due to their message ordering, low latency, and exactly-once delivery guarantees.
Using polyglot persistency with different data stores for read and write purposes can help solve the mismatch between write and read paths in microservices data management.
Implementing a backend rate limiter using Flink as a Kafka consumer can help prevent exhausting an external system (e.g., a database) due to high message arrival rates from Kafka.

Data automation and orchestration tools simplify data management tasks for ML applications.
These tools combine data from various sources, clean and transform it for specific ML algorithms.
The sector offers a broad range of tools, from ETL to specialized ML automation platforms, to cater to diverse data types and company needs.