The hottest Data Systems Substack posts right now

Single Tenant Architecture provides each customer their own independent database and software instance, offering security and customization like living in a detached house.
Multi-Tenant Architecture is akin to an apartment building where multiple tenants share common infrastructure, allowing for economies of scale but potentially limiting customization.
Single Tenant architecture is known for high user engagement, control, and stability, while Multi-Tenant architecture favors compliance, security, and quick onboarding for better scalability.

Learning to decompose a data system is crucial for better reasoning and understanding of large infrastructure
Decomposing a data system allows for scalability, identification of bottlenecks, and total event processing latency optimization
The different layers in a data system include data ingestion, transformation, and serving layers, each with specific functions and technologies

Watermarks in Stream Processing help handle event lateness and decide when to treat data as 'late data'.
In SQL Query execution, the order is FROM and JOIN, WHERE, GROUP BY, HAVING, SELECT, ORDER BY, LIMIT.
To optimize SQL Queries, reduce dataset sizes for joins and use subqueries for pre-filtering.

Data latency can unexpectedly increase at any stage of the Data Pipeline.
It is essential to decompose the Data System into smaller elements to identify and fix bottlenecks efficiently.
Effective monitoring of each component is necessary to avoid breaches in SLAs.

The difference between OLTP and OLAP systems can cause miscommunication among data producers and consumers.
OLTP systems focus on serving end users quickly with specific product features, while OLAP systems handle complex analytics by scanning large amounts of data.
Empathy and communication between OLTP and OLAP teams are crucial to building scalable data products.

Get a weekly roundup of the best Substack posts, by hacker news affinity: