The hottest Data Systems Substack posts right now

And their main takeaways
Category
Top Technology Topics

How health institutions can catch up

Your Local Epidemiologist • 1266 implied HN points • 17 Dec 25
🏥 Health Politics Data Systems
  1. Health institutions are behind the times and must change how they communicate by listening, being transparent, and meeting people where they are instead of broadcasting one-way guidance.
  2. Project Stethoscope / PHNIX shows a practical path forward: actively listen to communities, partner with trusted local messengers, and bring lived experiences into the systems that make decisions.
  3. Trust is not declared but earned through consistent, human actions like showing up, admitting mistakes, and acting promptly, and rebuilding that trust is essential for public health to protect people.

Life Update: Just Moved to SF Bay Area

clkao@substack • 99 implied HN points • 26 Aug 24
🕹 Technology Data Systems
  1. The move to the Bay Area was inspired by a feeling of belonging and the need for a supportive environment for their startup, Recce.
  2. Recce aims to improve the code review process for data-centric software development, addressing new challenges in correctness and testing.
  3. The writer appreciates the help from friends during the move and looks forward to sharing more about their experiences in this new chapter.

The CAP Theorem needed an update.

System Design Classroom • 359 implied HN points • 28 Apr 24
🕹 Technology Data Systems
  1. The CAP theorem says you can have consistency, availability, or partition tolerance, but only two at a time. This means systems have to make trade-offs depending on what they prioritize.
  2. The PACELC theorem expands on CAP by considering what happens during normal operation without network issues. It adds more options about choosing between latency and consistency.
  3. Real-world examples, like a multiplayer game leaderboard, show how these principles apply. You can have quick updates with potential outdated info or consistent scores that take longer to change.

All you need to know about the Google File System

VuTrinh. • 119 implied HN points • 11 May 24
🕹 Technology Data Systems
  1. Google File System (GFS) is designed to handle huge files and many users at once. Instead of overwriting data, it mainly focuses on adding new information to files.
  2. The system uses a single master server to manage file information, making it easier to keep track of where everything is stored. Clients communicate directly with chunk servers for faster data access.
  3. GFS prioritizes reliability by storing multiple copies of data on different chunk servers. It constantly checks for errors and can quickly restore lost or corrupted data from healthy replicas.

Beyond Big Tech: The Reality Of Data Engineering Outside Silicon Valley

SeattleDataGuy’s Newsletter • 847 implied HN points • 14 Dec 24
🕹 Technology Data Systems
  1. Working in big tech offers many advantages like better tools and a strong focus on data. This environment makes it easier to get work done quickly and efficiently.
  2. Many companies outside big tech struggle with data because it's not their main focus. They often use a mix of different tools that don't work well together, leading to confusion.
  3. Without a strong data leader, companies may find it hard to prioritize data spending. If data isn't tied to profits, it's tougher to justify investing time and money into it.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

artificial intelligence and urban planning | thinkthinkthink #26

thinkthinkthink • 65 implied HN points • 14 Nov 25
🕹 Technology Data Systems
  1. Urban planning needs to change from being static to adaptive. Cities should continuously adjust their planning based on current data and local needs.
  2. We can use technology like Large Language Models to break down cities into understandable parts. This helps us see how different elements of a city interact and affect each other.
  3. New governance systems such as Decentralized Autonomous Organizations allow for more community involvement in decision-making. This means people can vote based on how strongly they feel about an issue, leading to fairer outcomes.

SAI #22: Decomposing the Data System.

SwirlAI Newsletter • 294 implied HN points • 18 Mar 23
🕹 Technology Data Systems
  1. Learning to decompose a data system is crucial for better reasoning and understanding of large infrastructure
  2. Decomposing a data system allows for scalability, identification of bottlenecks, and total event processing latency optimization
  3. The different layers in a data system include data ingestion, transformation, and serving layers, each with specific functions and technologies

Single Tenant vs Multi Tenant Architecture for SAAS Platforms [System Design Sundays]

Technology Made Simple • 139 implied HN points • 04 Dec 23
🕹 Technology Data Systems
  1. Single Tenant Architecture provides each customer their own independent database and software instance, offering security and customization like living in a detached house.
  2. Multi-Tenant Architecture is akin to an apartment building where multiple tenants share common infrastructure, allowing for economies of scale but potentially limiting customization.
  3. Single Tenant architecture is known for high user engagement, control, and stability, while Multi-Tenant architecture favors compliance, security, and quick onboarding for better scalability.

A Halloween fright fest

Metal Machine Music by Ben Tarnoff • 59 implied HN points • 31 Oct 19
🕹 Technology Data Systems
  1. AI ethics initiatives are aiming to establish responsible rules for AI system development but can lack democratic input from those impacted by the technology.
  2. Democratizing AI involves making decisions about values political, requiring mechanisms for collective decision-making to ensure fairness and transparency in algorithmic processes.
  3. Kristen Nygaard, a Norwegian computer scientist, was instrumental in developing object-oriented programming and also worked to empower workers in their workplaces through understanding and influencing technology.

OLTP vs OLAP: The Core of Data Miscommunication

Data Products • 2 HN points • 23 Jun 23
🕹 Technology Data Systems
  1. The difference between OLTP and OLAP systems can cause miscommunication among data producers and consumers.
  2. OLTP systems focus on serving end users quickly with specific product features, while OLAP systems handle complex analytics by scanning large amounts of data.
  3. Empathy and communication between OLTP and OLAP teams are crucial to building scalable data products.

Data Replication 101: Strategies for Performance, Availability, and DR

DataSketch’s Substack • 0 implied HN points • 21 Feb 24
🕹 Technology Data Systems
  1. Data replication creates multiple copies of data to ensure it is always available and resilient against failures. This means if one server goes down, others can still keep running smoothly.
  2. There are different strategies for data replication like master-slave and multi-master setups. Each one has its own benefits, especially when it comes to how they handle read and write operations.
  3. Monitoring and tuning your replication setup is essential. By keeping an eye on performance and any issues, businesses can make sure their data systems run efficiently and reliably.

A First Principles guide to Data Availability: Part 2

Matthew’s Substack • 0 implied HN points • 31 Jul 24
🕹 Technology Data Systems
  1. Data Availability (DA) is crucial for ensuring that transaction data is accessible and secure, especially as blockchain technology grows. New solutions are needed to handle increased demand without high costs.
  2. There are two main types of DA solutions: Ordered DA, which includes consensus and provides stronger security, and DACs (Data Availability Committees), which focus on scalability and lower costs but offer less security.
  3. Choosing the right DA solution depends on factors like transaction value, data cost, and security needs. Different use cases, like finance or gaming, may prefer different DA features.

GroupBy #21: How to design resilient and large scale data systems, What Data Modeling is NOT

VuTrinh. • 0 implied HN points • 06 Feb 24
🕹 Technology Data Systems
  1. Designing data systems requires resilience and scalability, which means they should handle growth and failures efficiently.
  2. Data modeling is more than just making diagrams; it's about understanding the entire system and how data flows within it.
  3. Using tools like DuckDB in the browser can open up new possibilities for data processing, making it more accessible and flexible.