The hottest System Design Substack posts right now

And their main takeaways
Category
Top Technology Topics

Language isn't the cure

Sunday Letters • 119 implied HN points • 01 May 22
🕹 Technology System Design
  1. New programming languages or techniques won't solve old problems. Teams need to focus on fixing their systems before expecting a new language to make things better.
  2. People often believe learning a new language will improve their skills, but it doesn't address deeper issues like organization or trust. Just like learning a different language won’t make someone a great writer.
  3. Fixing systemic team issues takes hard work and discipline. Sometimes, sticking with familiar tools can be more effective than constantly chasing new options.

Resiliency beyond the Classic Circuit Breaker

System Design Classroom • 2 HN points • 10 Jul 24
🕹 Technology System Design
  1. To handle system failures, you can use different strategies like 'Fail Fast' which stops operations quickly to save resources. But this can affect user experience because they won't get a chance to recover from the error.
  2. Another approach is 'Fail Silent', where instead of showing an error, the system quietly returns a default value. It helps keep things running smoothly, but users might miss important information if data is missing.
  3. Lastly, there's 'Custom Fallback', which uses saved local data when a service fails. This keeps the service active, but the information might be outdated, which can confuse users.

How to code for the future

CodeFaster • 36 implied HN points • 19 Nov 24
🕹 Technology System Design
  1. When coding for the future, it's important not to create more work for yourself later. Focus on avoiding technical debt instead of trying to predict every future need.
  2. Don't go overboard with coding. Keep your code simple and flexible, ensuring it can adapt to changes without adding extra complexity.
  3. Instead of trying to build reusable programs from the start, solve the immediate problem first. You can refactor and create reusable parts later if needed.

Solve the right problem

Sunday Letters • 99 implied HN points • 27 Mar 22
🕹 Technology System Design
  1. It's important to solve the right problem. If you don't understand the problem clearly, you might end up overcomplicating things.
  2. Sometimes, less is more. Simplifying designs can make your code easier to manage and improve user experience.
  3. Don't try to fix every possible edge case; focus on the most common scenarios. This can help you avoid unnecessary complexity.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Rate Limiters[System Design Sundays]

Technology Made Simple • 59 implied HN points • 11 Apr 22
🕹 Technology System Design
  1. Rate limiters are crucial for controlling traffic and preventing system crashes.
  2. Different kinds of rate limiters like Request Rate Limiter and Concurrent Rate Limiter serve specific purposes.
  3. Rate limiters help protect against attacks, manage costs, and ensure high-priority tasks are handled efficiently.

Writeup System Design — Backend for Google Photos[Systems Design Sundays]

Technology Made Simple • 39 implied HN points • 24 Jul 22
🕹 Technology System Design
  1. To ace System Design Interviews, read/watch more system design mock interviews on YouTube, engineering blogs, and learn about important technologies/concepts.
  2. When designing a system like the backend for Google Photos, start with domain analysis, note requirements (functional and non-functional), expected load/performance, and user profiles for valuable solutions.
  3. Engage with the content you find helpful by liking, sharing, and offering feedback to creators. It helps them reach more people and improve their work.

Newsletter #12: System Design for Machine Learning - Part II

Decoding Coding • 19 implied HN points • 18 May 23
🕹 Technology System Design
  1. Airbnb uses a special tool called Zipline for feature engineering in their Customer Lifetime Value model, which helps them pick and create over 150 features needed for predictions.
  2. Chicisimo built a recommendation system based on user data, which includes both objective and subjective features, to give personalized fashion advice using their Social Fashion Graph.
  3. Case studies provide valuable lessons in applying frameworks to real-world projects, showing that you need both a good framework and experience from past projects to succeed.

The CAP Theorem [System Design Sundays]

Technology Made Simple • 39 implied HN points • 09 May 22
🕹 Technology System Design
  1. The CAP Theorem states that in a distributed system, you can only have two out of three characteristics: consistency, availability, and partition tolerance. It helps in choosing the right database solutions based on system needs.
  2. Understanding the CAP Theorem is crucial for Systems Design Interviews where you work on building distributed systems. It provides clarity on why different databases like MongoDB and Cassandra are chosen for specific scenarios.
  3. To grasp the concepts of the CAP Theorem and learn about suitable databases, watching educational videos and reading articles can be very beneficial.

How to Make Money in Machine Learning, without doing the Math/Theory[Finance Fridays]

Technology Made Simple • 39 implied HN points • 07 May 22
🕹 Technology System Design
  1. There are various ways to make money in Machine Learning beyond the traditional roles like AI research and Data Analysis, such as specializing in software engineering aspects like developing hardware, building data sources, creating pipelines, and designing platforms.
  2. Important skills to succeed in these alternative paths include writing good tests, mastering data compression and handling, and becoming proficient in large-scale system design to ensure scalability.
  3. Staying updated with ML resources and technologies like Airflow, Kubernetes, and Snowflake can be valuable for maximizing income opportunities in Machine Learning without needing to focus on the mathematics and theory aspects.

[System design] Sharding : In and Out Explained

The ZenMode • 42 implied HN points • 16 Mar 24
🕹 Technology System Design
  1. Sharding is a technique to horizontally partition a data store into smaller fragments across multiple servers, aiding in scalability and reliability.
  2. Before sharding a database, consider options like vertical partitioning, database optimization, replication, and caching to improve performance without the added complexity of sharding.
  3. Different sharding strategies like Hash Sharding, Range Sharding, and Directory-Based Sharding have unique considerations and advantages based on factors like data distribution, queries, and maintenance.

How Netflix handles it operations[System Design Sundays]

Technology Made Simple • 39 implied HN points • 04 Apr 22
🕹 Technology System Design
  1. Good systems are created using a top-down approach, focusing on important decisions with the highest ROI, especially in limited time/resource scenarios like interviews.
  2. Understanding recent tools and technologies is crucial for systems design, different from Leetcode-style problems.
  3. Thinking from multiple perspectives such as developer, investor, and user is key to mastering systems design, helping to tackle complex questions effectively.

New Newsletter Structure

Technology Made Simple • 39 implied HN points • 13 Mar 22
🚌 Education System Design
  1. The newsletter is now going daily, offering a comprehensive schedule for coding interview preparation.
  2. The schedule includes themed days like Math Monday and Technique Tuesday, covering important aspects of coding interviews.
  3. Premium subscribers gain access to in-depth solutions, stories, and system design discussions, enhancing their interview preparation.

How does Live Streaming Work [System Design Sundays]

Technology Made Simple • 19 implied HN points • 03 Oct 22
🕹 Technology System Design
  1. Systems design is complex and involves understanding intricacies built over years by many people.
  2. Avoid overwhelming yourself by trying to study everything when preparing for systems design interviews; focus on what you know, are interested in, or relates to your work.
  3. In systems design interviews, emphasize your expertise, ability to manage stakeholder needs, handle ambiguity, and foresee challenges; mentioning industry practices from engineering blogs can make you stand out.

Federated Learning [Systems Design Sundays]

Technology Made Simple • 19 implied HN points • 31 Jul 22
🕹 Technology System Design
  1. Federated Learning is a system where individual devices have their own mini ML models that update based on user input, then the updates are shared with a central server for a collective model.
  2. Federated Learning offers cheaper training and better security, making it a good solution for dynamic systems that constantly change based on user preferences.
  3. Federated Learning is beneficial for applications like social media or recommendation systems that require distributed learning processes and can handle multiple user input interfaces.

Undertstanding Kafka [System Design Sundays]

Technology Made Simple • 19 implied HN points • 03 Jul 22
🕹 Technology System Design
  1. Apache Kafka is an open-source distributed event streaming platform used for handling large amounts of data generated by multiple sources simultaneously.
  2. Kafka's key functions include publishing and subscribing to streams of records, storing records in order, and processing streams in real-time, making it essential for real-time streaming data pipelines and applications.
  3. Kafka offers 5 APIs (admin, producer, consumer, streams, connector) for managing topics, publishing streams, subscribing, implementing stream processing, and building data connectors, demonstrating its versatility and usability in system design.

The Time Premium

Kyla’s Newsletter • 42 implied HN points • 01 Jun 23
💰 Finance System Design
  1. Life is short and unpredictable, so cherish moments with loved ones
  2. Systems are intentionally designed to mislead and confuse, highlighting the importance of understanding underlying truths
  3. Vulnerability and self-reflection are crucial, especially in a world where trust is eroding

Design a Tic-Tac-Toe Game [with Code]

The ZenMode • 42 implied HN points • 15 Feb 23
🕹 Technology System Design
  1. Designing a Tic-Tac-Toe game involves creating a user interface with components like a game board, player information, game status, and start/restart buttons.
  2. Implementing game logic requires components like game state, move validation, win condition detection, tie condition detection, and player management.
  3. For scalability, maintainability, and extensibility, consider using components like load balancer, cloud-based infrastructure, modular architecture, clean code, and automated testing.

How would you design Twitter

Technology Made Simple • 19 implied HN points • 16 Mar 22
🕹 Technology System Design
  1. When designing a system like Twitter, it's important to approach the problem methodically and break it down into smaller sub-problems to find solutions.
  2. System design interviews don't have strict right or wrong answers, so it's okay to focus on different aspects based on your expertise.
  3. Engaging with resources and trying to solve the problem yourself first before looking at solutions can be a valuable learning approach for system design.

🥳 The Perils of Over-Abstraction, ts_sms Compression Tool, Kqueue vs. Epoll & More

ppdispatch • 5 implied HN points • 31 Dec 24
🕹 Technology System Design
  1. Over-abstraction in code can make things complicated and hard to manage, so it's important to keep it simple. If you complicate your system, it might end up slowing down and confusing your team.
  2. Fish-shell switched from C++ to Rust to improve safety and performance, showing how changing your tools can lead to better results. Their move has also engaged the community and made contributions easier.
  3. Understanding the differences between PHP's getenv() and $_ENV can prevent unexpected issues when accessing environment variables. It's essential to know how your PHP configuration handles these variables to avoid problems.

Premature Optimization and Rapid Prototyping

Load-bearing Tomato • 8 implied HN points • 23 Apr 24
🕹 Technology System Design
  1. Games need to start with quick experiments to see what works, this is called rapid prototyping. Flexibility is important so designers can try new ideas without being held back.
  2. Code doesn’t have to be perfect; it just needs to be good enough for what the game requires. Sometimes a simpler solution works best and saves time.
  3. It's crucial to know when to optimize code. If the game is running well and meeting its needs, there might not be a need to improve it right away.

Three domains in Data Science

Laszlo’s Newsletter • 16 implied HN points • 19 Apr 23
🕹 Technology System Design
  1. Domains in data science help break up complex systems for easier comprehension and focus.
  2. Boundaries between domains help prevent misunderstandings and allow for clear communication.
  3. Having clear separation of three domains in data science aids in assigning concerns correctly and focusing effectively.

Episode 18: Burkhard on Qt Embedded Systems

burkhardstubert • 19 implied HN points • 07 Jun 21
🕹 Technology System Design
  1. A good architecture meets three criteria: it should be technically sound, fulfill the needs of stakeholders, and deliver real value. Mixing different architectural patterns often results in better designs.
  2. Understanding and addressing operating conditions like temperature or dust is important in system architecture. These factors can influence design decisions considerably.
  3. It's crucial to balance modeling and implementation for successful architecture. Rely on both upfront design and iterative feedback from working code to adapt to changing requirements.

System Design of Uber's CacheFront - Serving more than 40 million low-latency reads per second

Engineering At Scale • 4 HN points • 03 Mar 24
🕹 Technology System Design
  1. Uber developed CacheFront, an integrated caching solution to overcome problems like maintenance overhead, reduced developer productivity, and region failovers caused by using Redis for caching
  2. Docstore's architecture includes a Control plane, Query Engine, and Storage Engine, with relevant responsibilities for each layer like query execution, data persistence, transaction management, and more
  3. CacheFront's design addressed non-functional requirements like consistency guarantees, cache warming & region failovers, fault tolerance, hot partition issues, and performance & cost improvements

System Design Fundamentals: What is a Load Balancer ?

Engineering At Scale • 2 HN points • 15 Jan 24
🕹 Technology System Design
  1. Load Balancers distribute client requests to different servers, improving system reliability and scalability.
  2. Load Balancers handle growing internet usage by evenly distributing workloads, preventing servers from being overwhelmed.
  3. Different types of Load Balancers include Hardware, Software, and Cloud Load Balancers, each with unique benefits for system optimization.

System Design Concepts: Dive deep into Database Sharding Strategies

Engineering At Scale • 2 HN points • 05 Aug 23
🕹 Technology System Design
  1. Range-Based Sharding divides data based on ranges like organizing books in bookshelves to make searches easier.
  2. Hash-Based Sharding evenly distributes data across different shards using a hash function, but may require data rebalancing when the number of shards changes.
  3. Consistent Hashing minimizes data movement when adding or removing shards, improving scalability while Geo-Based Sharding stores data close to users for better performance.

Microservices impair the productivity of high performing programmers

Sheep Code • 1 HN point • 10 Aug 23
🕹 Technology System Design
  1. Microservices can limit the productivity of high performing coders due to the complexity of dependencies across teams.
  2. In a microservices system, expertise shifts from individual coding skills to system design and team collaboration.
  3. For a 10x engineer in a microservices environment, success is more about harmonizing microservices and teams than individual coding prowess.

Guidelines for Chaos Engineering, Part 2

realkinetic • 0 implied HN points • 08 Sep 20
🕹 Technology System Design
  1. Identify critical systems before introducing chaos engineering to ensure the most impact on the business.
  2. Focus on testing critical components first, particularly those dealing with state, before moving on to less critical systems.
  3. Chaos engineering is an iterative process that should be performed in non-production environments first, with an aim towards ultimately testing in production.

Monday Memo #2

Andrew's Substack • 0 implied HN points • 26 Jun 23
🕹 Technology System Design
  1. Understand the core insight in questions from executives to address feedback effectively.
  2. When users consistently do something 'wrong,' it might be the system that needs adjustment, not the users.
  3. Quality of screen time matters more than quantity - evaluate how apps respect your time and attention.

Newsletter #11: System Design for Machine Learning - Part I

Decoding Coding • 0 implied HN points • 04 May 23
🕹 Technology System Design
  1. Before starting on a machine learning project, it's important to define clear goals and understand how ML can help achieve them.
  2. Setting up a data pipeline is crucial; it involves collecting, preparing, and analyzing data to see what features are useful for your model.
  3. When deploying machine learning models, you need to consider both hardware and software needs, including how to handle real-time data for ongoing training.

Objectivity and its Discontents

the best of a great lot • 0 implied HN points • 13 Sep 23
💼 Business System Design
  1. Data-driven decisionmaking in businesses aims to be objective by relying on numeric metrics to minimize human biases.
  2. Objectivity can create an illusion of completeness and can be manipulated to support pre-existing beliefs.
  3. Goodhart's Law warns that using measures as targets can lead to negative outcomes as individuals may prioritize meeting the measure over the original goal.

To Infinity and Beyond: Scaling Stream Processing

Bytewax • 0 implied HN points • 03 Oct 23
🕹 Technology System Design
  1. Bytewax has rescaling capabilities since version 0.17, allowing you to change the number of workers contributing to a dataflow cluster without losing data.
  2. Horizontal rescaling involves adding or removing workers from a cluster-based system to adjust computational resources.
  3. Bytewax utilizes state snapshots, primary assignment systems, and consistent routing to enable start-stop rescaling for streaming dataflows.

🧆 Newsletter #32: Rails Conf 2022, let Slack say "hummus", work with data...

Women On Rails Newsletter - International Version • 0 implied HN points • 24 May 22
🕹 Technology System Design
  1. Rails Conf 2022 had a positive impact with various community members giving talks and sharing knowledge. Supporting inclusive communities like WNB.rb is essential in tech.
  2. Using data effectively is crucial in tech - learn to tell compelling stories with data to justify technical decisions and report bugs convincingly.
  3. Explore topics like sharding data for scaling databases, choosing bug tracking tools wisely, and breaking down problems effectively for better problem-solving skills in tech.

Programming in C with an Assembler…

Andrew’s Substack • 0 implied HN points • 22 Oct 24
🕹 Technology System Design
  1. C is good for cross-platform development and handles important tasks like memory management well. This makes it easier for programmers to write efficient code.
  2. LM introduces modern programming features to C, like function templates and object-oriented programming styles. This can help make coding simpler and more powerful.
  3. The focus of LM is to tackle complex tasks that are hard in other languages, making it a valuable tool for systems programming. This means programmers can do more with less effort.

4 Things No One Tells You About Coding Interviews (and how you can improve today)

Brain Bytes • 0 implied HN points • 22 Nov 23
🕹 Technology System Design
  1. Studying data structures and algorithms is only one part of preparing for coding interviews; other essential aspects include communication skills, cultural fit, problem-solving ability, system design, and architecture.
  2. Technical interviews are not just about being technical but also about being a human who can solve technical problems; human skills like communication and cultural fit are crucial in interviews.
  3. Having strong communication skills, fitting into the company's culture, showcasing problem-solving abilities specifically in data structures and algorithms, and possessing knowledge in system design and architecture are key to excelling in coding interviews.