The hottest System Design Substack posts right now

And their main takeaways
Category
Top Technology Topics

Solve the right problem

Sunday Letters • 99 implied HN points • 27 Mar 22
🕹 Technology Software Development Engineering User Experience System Design Product Management
  1. It's important to solve the right problem. If you don't understand the problem clearly, you might end up overcomplicating things.
  2. Sometimes, less is more. Simplifying designs can make your code easier to manage and improve user experience.
  3. Don't try to fix every possible edge case; focus on the most common scenarios. This can help you avoid unnecessary complexity.

How Discord Handles Billions of Messages [System Design Sundays]

Technology Made Simple • 39 implied HN points • 17 Oct 22
🕹 Technology System Design Messaging Database Search Engineering
  1. Storage is crucial in system design, requiring detailed analysis of data generated for efficient handling.
  2. Discord uses Cassandra as its database to manage the storage of messages efficiently.
  3. Indexing messages and using tools like ElasticSearch are key for efficient search functionality in large data sets.

Writeup System Design — Backend for Google Photos[Systems Design Sundays]

Technology Made Simple • 39 implied HN points • 24 Jul 22
🕹 Technology System Design Backend Interviews Engineering
  1. To ace System Design Interviews, read/watch more system design mock interviews on YouTube, engineering blogs, and learn about important technologies/concepts.
  2. When designing a system like the backend for Google Photos, start with domain analysis, note requirements (functional and non-functional), expected load/performance, and user profiles for valuable solutions.
  3. Engage with the content you find helpful by liking, sharing, and offering feedback to creators. It helps them reach more people and improve their work.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Newsletter #12: System Design for Machine Learning - Part II

Decoding Coding • 19 implied HN points • 18 May 23
🕹 Technology Machine Learning Data science System Design Case Studies Software Development
  1. Airbnb uses a special tool called Zipline for feature engineering in their Customer Lifetime Value model, which helps them pick and create over 150 features needed for predictions.
  2. Chicisimo built a recommendation system based on user data, which includes both objective and subjective features, to give personalized fashion advice using their Social Fashion Graph.
  3. Case studies provide valuable lessons in applying frameworks to real-world projects, showing that you need both a good framework and experience from past projects to succeed.

The CAP Theorem [System Design Sundays]

Technology Made Simple • 39 implied HN points • 09 May 22
🕹 Technology System Design
  1. The CAP Theorem states that in a distributed system, you can only have two out of three characteristics: consistency, availability, and partition tolerance. It helps in choosing the right database solutions based on system needs.
  2. Understanding the CAP Theorem is crucial for Systems Design Interviews where you work on building distributed systems. It provides clarity on why different databases like MongoDB and Cassandra are chosen for specific scenarios.
  3. To grasp the concepts of the CAP Theorem and learn about suitable databases, watching educational videos and reading articles can be very beneficial.

How to Make Money in Machine Learning, without doing the Math/Theory[Finance Fridays]

Technology Made Simple • 39 implied HN points • 07 May 22
🕹 Technology Machine Learning Software Engineering System Design
  1. There are various ways to make money in Machine Learning beyond the traditional roles like AI research and Data Analysis, such as specializing in software engineering aspects like developing hardware, building data sources, creating pipelines, and designing platforms.
  2. Important skills to succeed in these alternative paths include writing good tests, mastering data compression and handling, and becoming proficient in large-scale system design to ensure scalability.
  3. Staying updated with ML resources and technologies like Airflow, Kubernetes, and Snowflake can be valuable for maximizing income opportunities in Machine Learning without needing to focus on the mathematics and theory aspects.

Design a Tic-Tac-Toe Game [with Code]

The ZenMode • 42 implied HN points • 15 Feb 23
🕹 Technology Design Programming System Design Game Development Web Development
  1. Designing a Tic-Tac-Toe game involves creating a user interface with components like a game board, player information, game status, and start/restart buttons.
  2. Implementing game logic requires components like game state, move validation, win condition detection, tie condition detection, and player management.
  3. For scalability, maintainability, and extensibility, consider using components like load balancer, cloud-based infrastructure, modular architecture, clean code, and automated testing.

How Netflix handles it operations[System Design Sundays]

Technology Made Simple • 39 implied HN points • 04 Apr 22
🕹 Technology System Design Engineering Interview Preparation Tech Tools
  1. Good systems are created using a top-down approach, focusing on important decisions with the highest ROI, especially in limited time/resource scenarios like interviews.
  2. Understanding recent tools and technologies is crucial for systems design, different from Leetcode-style problems.
  3. Thinking from multiple perspectives such as developer, investor, and user is key to mastering systems design, helping to tackle complex questions effectively.

Premature Optimization and Rapid Prototyping

Load-bearing Tomato • 8 implied HN points • 23 Apr 24
🕹 Technology Game Development Software Engineering Prototyping Coding Practices System Design
  1. Games need to start with quick experiments to see what works, this is called rapid prototyping. Flexibility is important so designers can try new ideas without being held back.
  2. Code doesn’t have to be perfect; it just needs to be good enough for what the game requires. Sometimes a simpler solution works best and saves time.
  3. It's crucial to know when to optimize code. If the game is running well and meeting its needs, there might not be a need to improve it right away.

New Newsletter Structure

Technology Made Simple • 39 implied HN points • 13 Mar 22
🚌 Education Interview Preparation Math Techniques Finance System Design
  1. The newsletter is now going daily, offering a comprehensive schedule for coding interview preparation.
  2. The schedule includes themed days like Math Monday and Technique Tuesday, covering important aspects of coding interviews.
  3. Premium subscribers gain access to in-depth solutions, stories, and system design discussions, enhancing their interview preparation.

How does Live Streaming Work [System Design Sundays]

Technology Made Simple • 19 implied HN points • 03 Oct 22
🕹 Technology Live Streaming System Design Social media
  1. Systems design is complex and involves understanding intricacies built over years by many people.
  2. Avoid overwhelming yourself by trying to study everything when preparing for systems design interviews; focus on what you know, are interested in, or relates to your work.
  3. In systems design interviews, emphasize your expertise, ability to manage stakeholder needs, handle ambiguity, and foresee challenges; mentioning industry practices from engineering blogs can make you stand out.

Federated Learning [Systems Design Sundays]

Technology Made Simple • 19 implied HN points • 31 Jul 22
🕹 Technology Machine Learning Data Privacy System Design
  1. Federated Learning is a system where individual devices have their own mini ML models that update based on user input, then the updates are shared with a central server for a collective model.
  2. Federated Learning offers cheaper training and better security, making it a good solution for dynamic systems that constantly change based on user preferences.
  3. Federated Learning is beneficial for applications like social media or recommendation systems that require distributed learning processes and can handle multiple user input interfaces.

Undertstanding Kafka [System Design Sundays]

Technology Made Simple • 19 implied HN points • 03 Jul 22
🕹 Technology Tech news System Design Cloud Computing Open Source Tools
  1. Apache Kafka is an open-source distributed event streaming platform used for handling large amounts of data generated by multiple sources simultaneously.
  2. Kafka's key functions include publishing and subscribing to streams of records, storing records in order, and processing streams in real-time, making it essential for real-time streaming data pipelines and applications.
  3. Kafka offers 5 APIs (admin, producer, consumer, streams, connector) for managing topics, publishing streams, subscribing, implementing stream processing, and building data connectors, demonstrating its versatility and usability in system design.

How would you design Twitter

Technology Made Simple • 19 implied HN points • 16 Mar 22
🕹 Technology System Design Coding Interviews YouTube
  1. When designing a system like Twitter, it's important to approach the problem methodically and break it down into smaller sub-problems to find solutions.
  2. System design interviews don't have strict right or wrong answers, so it's okay to focus on different aspects based on your expertise.
  3. Engaging with resources and trying to solve the problem yourself first before looking at solutions can be a valuable learning approach for system design.

System Design of Uber's CacheFront - Serving more than 40 million low-latency reads per second

Engineering At Scale • 4 HN points • 03 Mar 24
🕹 Technology System Design Caching Databases Scalability Integrations
  1. Uber developed CacheFront, an integrated caching solution to overcome problems like maintenance overhead, reduced developer productivity, and region failovers caused by using Redis for caching
  2. Docstore's architecture includes a Control plane, Query Engine, and Storage Engine, with relevant responsibilities for each layer like query execution, data persistence, transaction management, and more
  3. CacheFront's design addressed non-functional requirements like consistency guarantees, cache warming & region failovers, fault tolerance, hot partition issues, and performance & cost improvements

Episode 18: Burkhard on Qt Embedded Systems

burkhardstubert • 19 implied HN points • 07 Jun 21
🕹 Technology Software Architecture System Design Embedded Systems Programming
  1. A good architecture meets three criteria: it should be technically sound, fulfill the needs of stakeholders, and deliver real value. Mixing different architectural patterns often results in better designs.
  2. Understanding and addressing operating conditions like temperature or dust is important in system architecture. These factors can influence design decisions considerably.
  3. It's crucial to balance modeling and implementation for successful architecture. Rely on both upfront design and iterative feedback from working code to adapt to changing requirements.

System Design Fundamentals: What is a Load Balancer ?

Engineering At Scale • 2 HN points • 15 Jan 24
🕹 Technology System Design Algorithms Hardware Software
  1. Load Balancers distribute client requests to different servers, improving system reliability and scalability.
  2. Load Balancers handle growing internet usage by evenly distributing workloads, preventing servers from being overwhelmed.
  3. Different types of Load Balancers include Hardware, Software, and Cloud Load Balancers, each with unique benefits for system optimization.

System Design Concepts: Dive deep into Database Sharding Strategies

Engineering At Scale • 2 HN points • 05 Aug 23
🕹 Technology System Design Scalability Performance Data security
  1. Range-Based Sharding divides data based on ranges like organizing books in bookshelves to make searches easier.
  2. Hash-Based Sharding evenly distributes data across different shards using a hash function, but may require data rebalancing when the number of shards changes.
  3. Consistent Hashing minimizes data movement when adding or removing shards, improving scalability while Geo-Based Sharding stores data close to users for better performance.

Microservices impair the productivity of high performing programmers

Sheep Code • 1 HN point • 10 Aug 23
🕹 Technology Software Engineering Microservices System Design
  1. Microservices can limit the productivity of high performing coders due to the complexity of dependencies across teams.
  2. In a microservices system, expertise shifts from individual coding skills to system design and team collaboration.
  3. For a 10x engineer in a microservices environment, success is more about harmonizing microservices and teams than individual coding prowess.

Monday Memo #2

Andrew's Substack • 0 implied HN points • 26 Jun 23
🕹 Technology Software companies System Design
  1. Understand the core insight in questions from executives to address feedback effectively.
  2. When users consistently do something 'wrong,' it might be the system that needs adjustment, not the users.
  3. Quality of screen time matters more than quantity - evaluate how apps respect your time and attention.

Newsletter #11: System Design for Machine Learning - Part I

Decoding Coding • 0 implied HN points • 04 May 23
🕹 Technology Machine Learning System Design Data science Software Engineering Artificial Intelligence
  1. Before starting on a machine learning project, it's important to define clear goals and understand how ML can help achieve them.
  2. Setting up a data pipeline is crucial; it involves collecting, preparing, and analyzing data to see what features are useful for your model.
  3. When deploying machine learning models, you need to consider both hardware and software needs, including how to handle real-time data for ongoing training.

System Design Fundamentals: Understanding Scalability

Engineering At Scale • 0 implied HN points • 10 Jun 23
🕹 Technology System Design Scalability Cloud Computing Infrastructure Software Development
  1. Scalability is crucial for software systems to handle increasing demand and data.
  2. Building scalable systems can involve horizontal scaling (adding more machines) or vertical scaling (adding more resources to the same machine).
  3. Cloud technologies, like auto-scaling and managed databases, offer solutions for building scalable systems.

How to Design Metrics With Prometheus Metric Types: the USE Method

DevCube • 0 implied HN points • 21 Mar 23
🕹 Technology Metrics System Design CPU Monitoring
  1. The USE method for designing metrics focuses on Utilization, Errors, and Saturation of system resources.
  2. Implementing the USE method involves observing CPU, memory, and network metrics with tools like Prometheus and Grafana.
  3. CPU utilization can be calculated using metrics like node_cpu_seconds_total to understand how busy the CPU is.

Programming in C with an Assembler…

Andrew’s Substack • 0 implied HN points • 22 Oct 24
🕹 Technology Programming Software Computing Development System Design
  1. C is good for cross-platform development and handles important tasks like memory management well. This makes it easier for programmers to write efficient code.
  2. LM introduces modern programming features to C, like function templates and object-oriented programming styles. This can help make coding simpler and more powerful.
  3. The focus of LM is to tackle complex tasks that are hard in other languages, making it a valuable tool for systems programming. This means programmers can do more with less effort.

Splitwise - System Design [HLD]

The ZenMode • 0 implied HN points • 03 Nov 24
🕹 Technology System Design Software Development Mobile Applications User Experience
  1. Splitwise helps users track shared expenses easily. It lets people split costs for outings, so everyone knows what they owe.
  2. Users can create accounts, join groups, and add expenses that can be assigned to different members. The app automatically calculates what each person owes.
  3. The system is designed to handle many users securely while providing quick access to important information like balances and recent transactions.

HFC EP 006: Collapsing code paths

Hasen Judi • 0 implied HN points • 05 Jan 25
🕹 Technology Software Development Programming Web Development IT Infrastructure System Design
  1. There are two similar code paths for fetching posts by hashtag and user ID, causing duplication that can complicate the codebase. Simplifying these could make the code easier to manage.
  2. Using a single index for both types of queries can reduce complexity and allow for easier future features, like querying by other criteria, without increasing code duplication.
  3. Collapsing code paths not only streamlines current processes but also makes it easier to implement new features or queries in the future, reducing overall development effort.

System Design Mock Interview: Design TikTok ft. Google TPM[System Design Sundays]

Technology Made Simple • 0 implied HN points • 23 Jan 23
🕹 Technology System Design TikTok Tech industry
  1. When doing system design mock interviews, ask questions to clarify and understand the interviewer's values.
  2. Focus on your strengths and don't get overwhelmed by trying to know everything in system design interviews.
  3. Define a minimum viable product (MVP) that is linked to user experience in system design interviews.

🐿️ Bypassing Technical Debt - The Insanely Profitable Tech Newsletter

Squirrel Squadron Substack • 0 implied HN points • 14 Jan 25
🕹 Technology Software Development Technical Debt System Design Engineering Culture
  1. Avoid doing a total rewrite of your software, as it often leads to mistakes. Instead, make small, incremental changes to improve what you have.
  2. Technical debt is a common issue in tech. Learning to manage and refactor it can lead to better software over time.
  3. Just as our bodies have remnants of evolution, software can have old parts that still serve a purpose. It's important to understand their history before trying to remove them.

4 Things No One Tells You About Coding Interviews (and how you can improve today)

Brain Bytes • 0 implied HN points • 22 Nov 23
🕹 Technology Coding Communication Problem Solving System Design
  1. Studying data structures and algorithms is only one part of preparing for coding interviews; other essential aspects include communication skills, cultural fit, problem-solving ability, system design, and architecture.
  2. Technical interviews are not just about being technical but also about being a human who can solve technical problems; human skills like communication and cultural fit are crucial in interviews.
  3. Having strong communication skills, fitting into the company's culture, showcasing problem-solving abilities specifically in data structures and algorithms, and possessing knowledge in system design and architecture are key to excelling in coding interviews.

🧆 Newsletter #32: Rails Conf 2022, let Slack say "hummus", work with data...

Women On Rails Newsletter - International Version • 0 implied HN points • 24 May 22
🕹 Technology Web News Ruby Data System Design
  1. Rails Conf 2022 had a positive impact with various community members giving talks and sharing knowledge. Supporting inclusive communities like WNB.rb is essential in tech.
  2. Using data effectively is crucial in tech - learn to tell compelling stories with data to justify technical decisions and report bugs convincingly.
  3. Explore topics like sharding data for scaling databases, choosing bug tracking tools wisely, and breaking down problems effectively for better problem-solving skills in tech.

To Infinity and Beyond: Scaling Stream Processing

Bytewax • 0 implied HN points • 03 Oct 23
🕹 Technology Data processing Scaling System Design
  1. Bytewax has rescaling capabilities since version 0.17, allowing you to change the number of workers contributing to a dataflow cluster without losing data.
  2. Horizontal rescaling involves adding or removing workers from a cluster-based system to adjust computational resources.
  3. Bytewax utilizes state snapshots, primary assignment systems, and consistent routing to enable start-stop rescaling for streaming dataflows.

Objectivity and its Discontents

the best of a great lot • 0 implied HN points • 13 Sep 23
💼 Business Decision-making Performance Metrics System Design
  1. Data-driven decisionmaking in businesses aims to be objective by relying on numeric metrics to minimize human biases.
  2. Objectivity can create an illusion of completeness and can be manipulated to support pre-existing beliefs.
  3. Goodhart's Law warns that using measures as targets can lead to negative outcomes as individuals may prioritize meeting the measure over the original goal.

Guidelines for Chaos Engineering, Part 2

realkinetic • 0 implied HN points • 08 Sep 20
🕹 Technology Engineering Resilience System Design Monitoring
  1. Identify critical systems before introducing chaos engineering to ensure the most impact on the business.
  2. Focus on testing critical components first, particularly those dealing with state, before moving on to less critical systems.
  3. Chaos engineering is an iterative process that should be performed in non-production environments first, with an aim towards ultimately testing in production.