The hottest Distributed Systems Substack posts right now

And their main takeaways
Category
Top Technology Topics

InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper - Formerly InferenceMAX

SemiAnalysis • 10506 implied HN points • 16 Feb 26
đź•ą Technology Distributed Systems
  1. Nvidia’s Blackwell family (B200/B300/GB200/GB300) and NVL72 rack-scale systems deliver much higher inference throughput and far better tokens-per-dollar than prior Hopper GPUs, especially when paired with TensorRT-LLM, disaggregated prefill, and wide expert parallelism.
  2. AMD’s MI355X can be competitive on single-node FP8 SGLang setups, but its software stack struggles to compose FP4, disaggregated prefill, and wide EP together; AMD needs stronger upstream contributions, CI resources, and focus on composability to close the gap.
  3. Disaggregated prefill, wide expert parallelism, and multi-token prediction (MTP) are the key inference optimizations today, and when tuned against the throughput-vs-latency tradeoff they can massively lower cost per token while requiring accuracy checks to avoid silent regressions.

@adlrocha - Auto-research: The Lab that runs while you sleep

@adlrocha Weekly Newsletter • 64 implied HN points • 13 Mar 26
đź•ą Technology Distributed Systems
  1. A simple edit-evaluate-keep loop lets autonomous agents run short experiments and find real improvements by iterating quickly on a single editable training file and a fast proxy metric like validation bits-per-byte.
  2. Many small agents running on varied hardware can share discoveries via gossip protocols and turn idle or distributed GPUs into a decentralized research swarm that accelerates optimizations collectively.
  3. Picking the right evaluation and reward function is the hard part—designing clean, fast proxies and constraints (research taste) will matter more than raw execution in many fields, especially where feedback is slow or noisy.

The Most Complicated System I Ever Worked On

Jacob’s Tech Tavern • 1312 implied HN points • 17 Feb 26
đź•ą Technology Distributed Systems
  1. A single feature can balloon into a ludicrously elaborate pipeline that combines webscraping, long-running downloads, parsing and storage of large data, real-time analysis, and high-volume upload/polling.
  2. Most engineering work is routine, but rare peak challenges require orchestrating many moving parts and constant attention so they don’t overwhelm the team.
  3. Making a reliable system on top of unreliable third-party services takes sustained hardening and ongoing “whack-a-mole” maintenance to turn an MVP into production-grade software.

The Overview Of Apache Spark

VuTrinh. • 879 implied HN points • 07 Sep 24
đź•ą Technology Distributed Systems
  1. Apache Spark is a powerful tool for processing large amounts of data quickly. It does this by using many computers to work on the data at the same time.
  2. A Spark application has different parts, like a driver that directs processing and executors that do the work. This helps organize tasks and manage workloads efficiently.
  3. The main data unit in Spark is called RDD, which stands for Resilient Distributed Dataset. RDDs are important because they make data processing flexible and help recover data if something goes wrong.

NeXTWorld Interviews Bud Tribble, One of the NeXT Founders (1994)

Computer Ads from the Past • 1024 implied HN points • 01 Feb 26
đź•ą Technology Distributed Systems
  1. Sun picked NeXT’s OpenStep because it was a shipping, customer-tested object application environment that fit their distributed-object vision and gave a clear time-to-market advantage over building something new or waiting for competitors.
  2. OpenStep is being promoted as an industry standard through bodies like OMG and X/Open, but standardization will be gradual and will require proven implementations; it’s designed to work across languages and CORBA/IDL boundaries for interoperability.
  3. OpenStep will coexist with procedural environments and Windows compatibility on the same desktop, aiming for smooth interoperability (shared imaging, cut/copy/paste, and even a common Dock concept), while NeXT and Sun collaborate on ports and future evolution alongside licensing and platform sales.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Kafka can be so much more

Complexity is overrated • 85 implied HN points • 24 Feb 26
đź•ą Technology Distributed Systems
  1. Data should be viewed as a stream of events rather than just a static database state, and Kafka implements this by providing a distributed immutable commit log that decouples producers and consumers.
  2. Kafka is extremely versatile and gets used for many scenarios beyond its original use case, but teams often pigeonhole it or call it overkill for problems it can actually solve well.
  3. An expanding Kafka ecosystem (Kafka++) — integrating tools like Flink and Iceberg — makes real-time streaming data more useful for analytics, data engineering, and operational use cases, widening who can benefit from Kafka.

Queues offer more than orderly processing

System Design Classroom • 679 implied HN points • 02 Jul 24
đź•ą Technology Distributed Systems
  1. Queues help different parts of a system work independently. This means you can change one part without affecting the others, making updates easier.
  2. They improve a system's ability to handle more users at once. You can add more servers to take in requests without needing to instantly boost how fast they are processed.
  3. Queues also keep things running smoothly during busy times. They act like a waiting area, holding tasks so no work gets lost even if things get too hectic.

Connection Pooling: Fundamentals, Challenges and Trade-offs

Engineering At Scale • 795 implied HN points • 29 Nov 25
đź•ą Technology Distributed Systems
  1. Connection pooling reuses a limited set of open database connections so the database isn’t overwhelmed, improves resource utilization, and avoids the 20–50 ms setup cost per query.
  2. Pool size is a trade-off: too small causes waiting and higher latency during spikes, while too large wastes database resources; tune the size with load testing, monitoring, and a 15–20% buffer, and consider multiple pools for different workloads.
  3. Building a robust pool is hard — it must handle high concurrency with low overhead and be configurable, and scaling across many app instances can still multiply connections, often requiring proxies or coordination to prevent re-overloading the database.

Diving Deep into LinkedIn's Data Infrastructure: My 6-Hour Learning & Key Takeaways

VuTrinh. • 299 implied HN points • 03 Aug 24
đź•ą Technology Distributed Systems
  1. LinkedIn's data infrastructure is organized into three main tiers: data, service, and display. This setup helps the system to scale easily without moving data around.
  2. Voldemort is LinkedIn's key-value store that efficiently handles high-traffic queries and allows easy scaling by adding new nodes without downtime.
  3. Databus is a change data capture system that keeps LinkedIn's databases synchronized across applications, allowing for quick updates and consistent data flow.

Apache Kafka - Overview

VuTrinh. • 539 implied HN points • 06 Jul 24
đź•ą Technology Distributed Systems
  1. Apache Kafka is a system for handling large amounts of data messages, making it easier for companies like LinkedIn to manage and analyze user activity and other important metrics.
  2. In Kafka, messages are organized into topics and divided into partitions, allowing for better performance and scalability. This way, different servers can handle parts of the data at once.
  3. Kafka uses a pull model for consumers, meaning they can request data as they need it. This helps prevent overwhelming the consumers with too much data at once.

Involutions: The License to Update in O(1)

Software Bits Newsletter • 206 implied HN points • 14 Jan 26
đź•ą Technology Distributed Systems
  1. XOR is an involution: applying the same XOR twice cancels it out, so adding and removing an element use the same operation and let you update combined hashes in O(1).
  2. Zobrist hashing leverages XOR to update a chessboard hash with only a few XORs per move, enabling fast transposition-table lookups and huge search speedups; collisions are possible but usually acceptable or verifiable.
  3. More generally, pick the algebraic tool that matches your needs — use involutions like XOR for O(1) incremental updates when collisions are tolerable, rolling linear hashes for sliding windows, or Merkle trees when cryptographic integrity is required.

Apache Kafka - Important Designs

VuTrinh. • 259 implied HN points • 13 Jul 24
đź•ą Technology Distributed Systems
  1. Kafka uses the operating system's filesystem to store data, which helps it run faster by leveraging the page cache. This avoids the need to keep too much data in memory, making it simpler to manage.
  2. The way Kafka reads and writes data is done in a sequential order, which is more efficient than random access. This design improves performance, as accessing data in a sequence reduces delays.
  3. Kafka groups messages together before sending them, which helps reduce the number of requests made to the system. This batching process improves performance by allowing larger, more efficient data transfers.

Apache Kafka - Producer

VuTrinh. • 199 implied HN points • 20 Jul 24
đź•ą Technology Distributed Systems
  1. Kafka producers are responsible for sending messages to servers. They prepare the messages, choose where to send them, and then actually send them to the Kafka brokers.
  2. There are different ways to send messages: fire-and-forget, synchronous, and asynchronous. Each method has its pros and cons, depending on whether you want speed or reliability.
  3. Producers can control message acknowledgment with the 'acks' parameter to determine when a message is considered successfully sent. This parameter affects data safety, with options that range from no acknowledgment to full confirmation from all replicas.

Setting the Record Straight: Debunking Anatoly's Misconceptions about Security in Distributed Ledger Networks

Dan Hughes • 339 implied HN points • 08 Jun 24
đź”® Crypto Distributed Systems
  1. The honest majority assumption is key for blockchain security. It means that most participants must act honestly to keep the network safe from attacks.
  2. Full nodes rely on validator nodes to check the validity of transactions. If most validators are dishonest, full nodes cannot prevent issues like double spending.
  3. Economic security is important for discouraging attacks on a network. High stakes for validators make it less likely for them to act maliciously, as the potential losses from being caught far outweigh any gains.

Database Proxies: Challenges, Working and Trade-offs

Engineering At Scale • 195 implied HN points • 13 Dec 25
đź•ą Technology Distributed Systems
  1. Database proxies sit between services and the database and multiplex many client connections onto a fixed pool of database connections, preventing connection spikes and making horizontal scaling safer.
  2. Proxies can add features like query caching, read/write routing, and sharding/replica management, which simplifies application logic and abstracts database topology from the app.
  3. Using a proxy comes with costs — extra deployment and maintenance overhead and added latency (~10–15 ms) — so they’re valuable for complex setups (replication, sharding, FaaS) but can be overkill for a single simple database and must be designed to avoid becoming a SPOF.

Commutativity: Why Transformers Need Positional Encodings

Software Bits Newsletter • 103 implied HN points • 01 Jan 26
đź•ą Technology Distributed Systems
  1. Self-attention treats all positions symmetrically, so permuting tokens just permutes outputs; because attention is permutation‑equivariant, Transformers need positional encodings to learn token order.
  2. Commutativity is a deliberate design trade‑off: it enables parallelization and is perfect for unordered data like point clouds, sets, and graphs, but it destroys order information so you must use non‑commutative models or inject positions when order matters (language, time series).
  3. Commutativity shows up across ML: global pooling gives useful invariance but loses location, gradient aggregation and distributed training rely on commutative sums, and floating‑point associativity issues can still cause small nondeterminism.

How does Uber handle petabytes of Spark shuffle data every day?

VuTrinh. • 159 implied HN points • 22 Jun 24
đź•ą Technology Distributed Systems
  1. Uber uses a Remote Shuffle Service (RSS) to handle large amounts of Spark shuffle data more efficiently. This means data is sent to a remote server instead of being saved on local disks during processing.
  2. By changing how data is transferred, the new system helps reduce failures and improve the lifespan of hardware. Now, servers can handle more jobs without crashing and SSDs last longer.
  3. RSS also streamlines the process for the reduce tasks, as they now only need to pull data from one server instead of multiple ones. This saves time and resources, making everything run smoother.

Beyond Snowflake: A Problem-Solver’s Guide to Distributed ID Generation

Software Bits Newsletter • 51 implied HN points • 04 Jan 26
đź•ą Technology Distributed Systems
  1. Memory allocator patterns — like per-node caches, hierarchical range grants, batching, and prefetching — transfer cleanly to distributed ID generation and let services hand out unique IDs locally with almost no coordination.
  2. There is no one-size-fits-all ID strategy: slabs and hierarchical ranges give extreme throughput and B-tree locality at the cost of wasted IDs and weaker global ordering, consensus gives strict global ordering and durability but costs latency and availability, and Snowflake-style schemes sit in between.
  3. The best engineering move is methodological: spot a related solved problem, extract its core principles (hierarchy, locality, batching, prefetching), and adapt them while accounting for distributed realities like partial failure and unbounded latency.

Finding the CAP theorem of agents

Sunday Letters • 59 implied HN points • 02 Jun 24
đź•ą Technology Distributed Systems
  1. The CAP theorem shows that in any distributed system, you can only achieve two out of three things: consistency, availability, or partition tolerance. This means when things go wrong, you have to choose which one you're willing to sacrifice.
  2. In AI programming, there's a similar tension between using complex AI models and the need for reliable, deterministic code. Balancing these two aspects is a challenge, much like the early challenges with web applications.
  3. As technology evolves, the understanding and frameworks around these issues may improve. Just like how programmers now design around the CAP theorem, we might see better solutions and choices for AI challenges in the future.

A glimpse of Apache Pinot, the real-time OLAP system from LinkedIn

VuTrinh. • 99 implied HN points • 30 Mar 24
đź•ą Technology Distributed Systems
  1. Apache Pinot is a real-time OLAP system developed by LinkedIn that allows for fast analytics on large sets of data. It can handle tens of thousands of analytical queries per second while providing near-instant results.
  2. The architecture is divided into key components like controllers, brokers, and servers which work together to process queries and manage data efficiently. Pinot is designed to quickly ingest and query fresh data from various sources, ensuring low latency.
  3. Pinot supports various indexing strategies, like star-tree indexes, to optimize complex queries. This enables faster query responses by pre-aggregating data, making it easier to analyze large volumes of information.

Global Incremental ID

Hung's Notes • 79 implied HN points • 13 Dec 23
đź•ą Technology Distributed Systems
  1. Global Incremental IDs are important for preventing ID collisions in distributed systems, especially during tasks like data backup and event ordering.
  2. UUID and Snowflake ID are two common types of global IDs, each with unique advantages and disadvantages. For instance, UUIDs are larger but widely used, while Snowflake IDs are smaller but more complex to generate.
  3. Different systems, like Sonyflake and Tinyid, offer specialized methods for generating IDs, helping to ensure performance and avoiding database bottlenecks.

Cursor vs Anthropic multi-agent coding experiments

Peter’s Substack • 2 implied HN points • 06 Feb 26
đź•ą Technology Distributed Systems
  1. Use a hierarchical decomposition where high-level planners break goals into subplanners and isolated workers so complex coding tasks are split, owned, and driven to completion recursively.
  2. Coordination and correctness are the main bottlenecks for parallel agents: naive locking and expecting perfect commits cause conflicts and serialization, so robust coordination and tolerance for imperfect commits are needed to scale.
  3. Human input still matters a lot—clear, prioritized instructions, tests, and failure analysis are essential to guide agents, enforce performance and resource limits, and catch subtle bugs agents miss.

Scaling Distributed Systems with the Scatter-Gather Pattern

Engineering At Scale • 60 implied HN points • 15 Feb 25
đź•ą Technology Distributed Systems
  1. The Scatter-Gather pattern helps speed up data retrieval by splitting requests to multiple servers at once, rather than one after the other. This makes systems respond faster, especially when lots of data is needed.
  2. Using this pattern can improve system efficiency by preventing wasted time waiting for responses from each service. This means the system can handle more requests at once.
  3. However, implementing Scatter-Gather can be tricky. It requires careful handling of errors and managing different data sources to ensure the information is accurate and reliable.

SRE book notes: Wrapping Up

Bit Maybe Wise • 39 implied HN points • 13 Apr 23
đź•ą Technology Distributed Systems
  1. The SRE book includes useful additional materials like availability tables and best practices.
  2. The book covers a wide range of topics related to Site Reliability Engineering.
  3. It emphasizes key practices such as embracing risk, eliminating toil, and effective troubleshooting.

Merging Can Be Made Eventually Consistent

Bram’s Thoughts • 19 implied HN points • 18 Dec 23
đź•ą Technology Distributed Systems
  1. In distributed version control, there's a way to ensure consistent merging regardless of the order merges are done.
  2. File states can be represented as a set of line positions with generation counts to determine the winning state during merging.
  3. Handling conflicts in merging requires presenting changes in the order they'll appear to everyone, not based on 'local' or 'remote' changes.

Deterministic Simulation Testing - Antithesis

Why Now • 1 implied HN point • 20 Jan 26
đź•ą Technology Distributed Systems
  1. Deterministic simulation testing runs your entire distributed system inside virtual machines controlled by a deterministic hypervisor so each test run is reproducible. It replaces wall-clock time with instruction-count-based virtual time so timing-dependent bugs can be replayed exactly.
  2. The platform combines property-based testing, fuzzing, and fault injection to automatically explore many scenarios and surface rare race conditions. All tests run in sandboxed clones of production so you can inject network blips and failures without risking real users.
  3. Determinism is achieved with techniques like single-core execution, intercepted time calls, and deterministic I/O plus numerous micro-optimizations. The outcome is precise, replayable failures that make debugging and fixing distributed-system bugs much easier.

A Gentle Introduction to Kafka API

Tributary Data • 1 HN point • 16 Apr 24
đź•ą Technology Distributed Systems
  1. Kafka started at LinkedIn and later evolved into Apache Kafka, maintaining its core functionalities. Various vendors offer their versions of Kafka but ensure the Kafka API remains consistent for compatibility.
  2. Apache Kafka acts as a distributed commit log storing messages in fault-tolerant ways, while the Kafka API is the interface used to interact with Kafka for reading, writing, and administrative operations.
  3. Kafka's structure involves brokers forming clusters, messages with keys and values, topics grouping messages, partitions dividing topics, and replication for fault tolerance. Understanding these architectural components is vital for working effectively with Kafka.

A Deep Dive into the Underlying Architecture of Groq's LPU

Confessions of a Code Addict • 4 HN points • 01 Mar 24
đź•ą Technology Distributed Systems
  1. Groq's LPU showcases an innovative design departing from traditional architectures, focusing on deterministic execution for enhanced performance.
  2. The TSP architecture achieves determinism through a simplified hardware design, enabling precise scheduling by compilers for predictable performance.
  3. Groq's approach to creating a distributed multi-TSP system eliminates non-determinism typical in networked systems, with the compiler efficiently managing data movement.

AI Pitfalls and Promises: Surprising New Findings in Cognitive Models

ppdispatch • 2 implied HN points • 01 Nov 24
đź•ą Technology Distributed Systems
  1. Chain-of-thought prompting might actually make some tasks harder for AI, especially in visual tasks where less thinking works better.
  2. The DAWN framework allows AI agents to work together globally in a secure way, which can lead to improved collaboration.
  3. New mesomorphic networks are great for understanding tabular data and give clearer explanations, making them useful for various applications.

[ChatGPT-generated] Demystifying Distributed Systems: Unveiling the Inner Workings

PseudoFreedom • 5 implied HN points • 26 May 23
đź•ą Technology Distributed Systems
  1. Distributed systems use interconnected computers to work as one unit, enhancing performance and scalability.
  2. Challenges in distributed systems include network communication, data consistency, and fault tolerance.
  3. Benefits of distributed systems include scalability, high availability, and improved performance through collective computing.

What's a distributed system?

Brick by Brick • 0 implied HN points • 05 Mar 24
đź•ą Technology Distributed Systems
  1. A distributed system is a collection of components on multiple computers that appear as a single, unified system to users. They are commonly used in database and file systems.
  2. Key characteristics of distributed systems include concurrency, scalability, fault tolerance, and decentralization, enabling efficient operation across multiple machines.
  3. In distributed systems, concepts like fault tolerance, recovery & durability, the CAP theorem, and quorums & consensus are crucial for maintaining reliability, consistency, and coordination among nodes.

Range Partitioning: Zero to One

aspiring.dev • 0 implied HN points • 17 Mar 24
đź•ą Technology Distributed Systems
  1. Range partitioning splits data into key ranges to improve performance and scalability. This method helps databases manage heavy loads by distributing data efficiently.
  2. Unlike hash partitioning, range partitioning allows for easier scaling. You can adjust the number of ranges as needed without the hassle of rewriting data.
  3. While range partitioning is powerful, it can be tricky to implement and may struggle with very sequential workloads. Planning is necessary to avoid creating performance hotspots.

The Future of Cloud Computing

Splattern • 0 implied HN points • 23 Dec 23
đź•ą Technology Distributed Systems
  1. Big tech cloud companies like AWS, Azure, and Google Cloud don't really foster innovation. They were built on existing technology, and their focus is more on business strategies than improving their tech.
  2. These companies have lost many of their original experienced employees. This means current workers might not have the skills needed to innovate in a fast-moving tech world.
  3. Startups are emerging with new models that can offer better pricing and solutions for cloud computing. This could threaten the big tech clouds and change the landscape of cloud services.

GroupBy #10: Netflix's Psyberg, Parquet format, SQL is not Designed for Analytics

VuTrinh. • 0 implied HN points • 21 Nov 23
đź•ą Technology Distributed Systems
  1. Netflix's Psyberg is a new way for processing data that helps manage membership information better. It uses innovative methods to make data processing more efficient.
  2. The Parquet format is great for storing data because it organizes information in a smart way. It can improve how quickly and easily data is accessed and processed.
  3. SQL isn't the best tool for doing analytics because it was designed a long time ago. There are newer tools that fit analytics needs much better.

Mastering Apache Spark Performance: A Data Engineer's Guide to Optimization

DataSketch’s Substack • 0 implied HN points • 14 Oct 24
đź•ą Technology Distributed Systems
  1. Properly configuring resources in Spark is really important. Make sure you adjust settings like memory and cores to fit your cluster's total resources.
  2. Good data partitioning helps Spark job performance a lot. For example, repartitioning your data based on a relevant column can lead to faster processing times.
  3. Using broadcast joins can save time and reduce workload. When joining smaller tables, broadcasting can make the process much quicker.