The hottest Machine Learning Substack posts right now

And their main takeaways
Category
Top Business Topics

Data Science Weekly - Issue 547

Data Science Weekly Newsletter • 179 implied HN points • 17 May 24
đź•ą Technology Machine Learning
  1. Learning Rust programming can be made easy with exercises designed for beginners, even if you know another language already. You’ll work through small tasks to build confidence.
  2. Data scientists need to learn how to work with databases to scale their analytics. Many face challenges when transitioning to this part of their work.
  3. There are helpful tools, like Data Wrangler for VS Code, that simplify data cleaning and analysis. These tools help generate code automatically as you work with your data.

Moral puzzles: Man vs. machine

DYNOMIGHT INTERNET NEWSLETTER • 562 implied HN points • 19 Jun 25
đź•ą Technology Machine Learning
  1. Current AI can understand human values to some extent, but it may not cover all complex situations. It's crucial to keep testing AI's responses on moral questions.
  2. People's opinions on moral dilemmas can vary significantly, especially on more unusual scenarios. This highlights the complexity of human ethics.
  3. Readers recognized that their views might differ from the general population, showing self-awareness in moral reasoning. It's good to be mindful of how diverse perspectives can be.

LabGenius - the long slow road to faster drug discovery

Rory’s Always On Newsletter • 535 implied HN points • 07 Feb 24
🔬 Science Machine Learning
  1. AI and machine learning are revolutionizing drug discovery by speeding up the identification of potential treatments, leading to big rewards for those in the industry.
  2. Building a successful biotech company requires patience, determination, and significant funding, often with a focus on research and development before revenue generation.
  3. Investors in biotech companies must be prepared for a long journey of constant failures and successes, akin to the process of drug discovery, with potential acquisitions being key outcomes.

Some ideas for what comes next

Democratizing Automation • 529 implied HN points • 23 Jun 25
đź•ą Technology Machine Learning
  1. OpenAI's new model, o3, is really good at finding information quickly, like a determined search dog. It's unique compared to other models, and many are curious if others will match its capabilities soon.
  2. AI agents, like Claude Code, are improving quickly and can solve complex tasks. They have made many small changes that boost their performance, which is exciting for users.
  3. The trend in AI models is slowing down in terms of size but improving in efficiency. Instead of just making bigger models, companies are focusing on optimizing what they already have.

Data Science Weekly - Issue 541

Data Science Weekly Newsletter • 279 implied HN points • 05 Apr 24
đź•ą Technology Machine Learning
  1. AI agents have unique challenges that traditional laws may not effectively solve. New rules and systems are needed to ensure they are managed properly.
  2. JS-Torch is a new JavaScript library that makes deep learning easier for developers familiar with PyTorch. It allows building and training neural networks directly in the browser.
  3. Data acquisition is crucial for AI start-ups to succeed. There are strategies outlined to help these businesses gather the right data efficiently.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

The AI safety problem is wanting

DYNOMIGHT INTERNET NEWSLETTER • 531 implied HN points • 26 Jun 25
đź•ą Technology Machine Learning
  1. AI safety is a big concern, and the main challenge is to make AI systems want to be nice to us. If they don't want to, they won't care about what we want.
  2. Trying to impose restrictions on AI won't work because a smarter AI can always find a way around them. Instead, we need to align AI with our values so it chooses to act positively.
  3. If we can ensure that AI genuinely wants to do what's best for us, the rest of the alignment problems become easier to manage. It's all about making sure AI understands and respects our values.

GroupBy #41: Uber’s Batch Data Infrastructure with Google Cloud Platform

VuTrinh. • 99 implied HN points • 25 Jun 24
đź•ą Technology Machine Learning
  1. Uber is moving its huge amount of data to Google Cloud to keep up with its growth. They want a smooth transition that won't disrupt current users.
  2. They are using existing technologies to make sure the change is easy. This includes tools that will help keep data safe and accessible during the move.
  3. Managing costs is a big concern for Uber. They plan to track and control spending carefully as they switch to cloud services.

No Free Dessert in Machine Learning

Mindful Modeler • 399 implied HN points • 20 Feb 24
🔬 Science Machine Learning
  1. Generalization in machine learning is essential for a model to perform well on unseen data.
  2. There are different types of generalization in machine learning: from training data to unseen data, from training data to application, and from sample data to a larger population.
  3. The No Free Lunch theorem in machine learning highlights that assumptions and effort are always needed for generalization, and there's no free lunch when it comes to achieving further generalization.

Finally—letters in Scroll 4!

Vesuvius Challenge • 64 implied HN points • 21 Dec 25
🔬 Science Machine Learning
  1. A new high-resolution tomographic scan (2.4 µm pixels, 78 keV, 22 cm propagation) revealed 5–6 mm letters in PHerc. 1667 that were invisible in earlier 8 µm scans.
  2. A generalist ink-detection model trained on other fragments detected letters immediately without scroll-specific labeling, suggesting the method can find ink across different scrolls.
  3. The team is retiring the First Letters and First Title prizes to focus on extracting text, and they doubled the Kaggle competition prize pool to $200,000 while preparing an updated dataset.

The Sequence Opinion #786: The Great Absorption: When System Code Becomes Model Weights

TheSequence • 56 implied HN points • 08 Jan 26
đź•ą Technology Machine Learning
  1. Many system and agent capabilities that used to live in external orchestration code are being internalized into model weights, so models now handle tasks once implemented by separate scripts and pipelines.
  2. Hand‑coded scaffolding like prompt chains, vector DB glue, and custom parsers is increasingly at risk of becoming obsolete whenever a new frontier model checkpoint appears, so expect rapid disruption.
  3. Product teams need to distinguish permanent infrastructure from temporary scaffolding and architect systems to tolerate or embrace model internalization, or else large parts of their stack can be replaced overnight.

AGI Is Already Here—It’s Just Not Evenly Distributed

The Algorithmic Bridge • 1104 implied HN points • 05 Feb 25
đź•ą Technology Machine Learning
  1. Understanding how to create good prompts is really important. If you learn to ask questions better, you'll get much better answers from AI.
  2. Even though AI models are getting better, good prompting skills are becoming more important. It's like having a smart friend; you need to know how to ask the right questions to get the best help.
  3. The better your prompting skills, the more you'll be able to take advantage of AI. It's not just about the AI's capabilities but also about how you interact with it.

Neural Network Fail Compilation

Top Carbon Chauvinist • 59 implied HN points • 21 Jul 24
đź•ą Technology Machine Learning
  1. AI systems, like large language models, struggle with reasoning and can often give wrong answers to simple questions. They rely on patterns rather than true understanding.
  2. Generative AI can produce flawed code and lead to increased mistakes in programming. This raises concerns about the overall quality and security of software.
  3. AI tools can create misleading or totally false news articles. Their results can be unreliable, which poses risks when using them for information or news reporting.

On OpenAI's Model Spec 2.0

Don't Worry About the Vase • 985 implied HN points • 21 Feb 25
đź•ą Technology Machine Learning
  1. OpenAI's Model Spec 2.0 introduces a structured command chain that prioritizes platform rules over individual developer and user instructions. This hierarchy helps ensure safety and performance in AI interactions.
  2. The updated rules emphasize the importance of preventing harm while still aiming to assist users in achieving their goals. This means the AI should avoid generating illegal or harmful content.
  3. There are notable improvements in clarity and detail compared to previous versions, like defining what content is prohibited and reinforcing user privacy. However, concerns remain about potential misuse of the system by those with access to higher-level rules.

The Sequence Radar #791: The Eastern Surge & The Silicon Valley Shuffle

TheSequence • 42 implied HN points • 18 Jan 26
đź•ą Technology Machine Learning
  1. Engram shows that offloading static facts to a huge O(1) lookup memory lets neural experts focus on reasoning, and allocating roughly 20–25% of sparse parameters to that memory hits an optimal loss curve.
  2. Chinese labs are rapidly closing the gap with stronger unified multimodal architectures like Baidu’s Ernie 5, and Zhipu’s GLM-Image—trained entirely on Huawei Ascend chips—demonstrates domestic hardware can support SOTA training runs.
  3. Talent is extremely scarce and fiercely contested, evidenced by rapid co-founder departures and rehires, while large bets on non-invasive brain-computer interfaces signal a push to boost human-AI bandwidth beyond typed text.

What I've been reading (#1)

Democratizing Automation • 490 implied HN points • 21 Jun 25
đź•ą Technology Machine Learning
  1. Links are important and will now have their own dedicated space. This way, they can be shared and discussed more easily.
  2. AI is being used more than many realize, and there's promising growth in its revenue. The future looks positive for those already in the industry.
  3. It's crucial to stay informed about advancements in AI, especially regarding human-AI relationships and the challenges that come with making AI more capable.

Data Science Weekly - Issue 543

Data Science Weekly Newsletter • 219 implied HN points • 19 Apr 24
đź•ą Technology Machine Learning
  1. Statistical ideas have a big impact on the world. Learning about important papers can help us understand how statistics shape modern research and decision-making.
  2. Machine Learning teams have different roles that face unique challenges. Understanding these personas can help leaders support their teams better.
  3. Using vector embeddings can greatly improve search experiences in apps. They simplify processes that previously seemed too complex and highlight their usefulness in technology.

How to build your first LLM evaluation

The AI Frontier • 159 implied HN points • 16 May 24
đź•ą Technology Machine Learning
  1. AI needs to show real value to its customers, which means proving it can create real profits. Without this, it’s hard to justify the excitement around AI.
  2. To understand how well AI products perform, it’s important to create custom evaluations that target specific goals. Generic measurements like MMLU don't provide useful insights for particular applications.
  3. Improving AI evaluations is a continuous process that requires careful scoring and can benefit from community feedback. It's crucial to identify weaknesses and refine metrics for more accurate assessments.

Don't "fix" your imbalanced data

Mindful Modeler • 818 implied HN points • 05 Sep 23
đź•ą Technology Machine Learning
  1. Avoid trying to fix imbalanced data through sampling methods like oversampling or undersampling. It can distort your model's calibration and reduce information for the majority class.
  2. SMOTE, a common method for imbalanced data, works well only with weak classifiers, not strong ones. It may not be suitable if calibration is crucial for your model.
  3. Consider doing nothing when faced with imbalanced data as a default strategy. Sometimes in machine learning, less is more.

Import AI 353: AI bootstrapping; LLMs as inventors; Facebook releases a free moderation tool

Import AI • 559 implied HN points • 18 Dec 23
đź•ą Technology Machine Learning
  1. AI bootstrapping is advancing, with techniques like ReST^EM by Google DeepMind showing ways to make models smarter iteratively.
  2. Language models like LLMs are being used for groundbreaking tasks, such as extending human knowledge through techniques like FunSearch by DeepMind.
  3. Facebook has released a free moderation LLM, Llama Guard, highlighting the use of powerful models to control and monitor outputs of other AI systems.

to kaggle, or not to kaggle

Mindful Modeler • 379 implied HN points • 13 Feb 24
đź•ą Technology Machine Learning
  1. There are conflicting views on Kaggle - some see it as a playground while others believe it produces top machine learning results.
  2. Participating in Kaggle competitions can be beneficial to learn core supervised machine learning concepts.
  3. The decision to focus on Kaggle competitions should depend on how much daily tasks align with Kaggle-style work.

AI and the — em dash

Technically • 28 implied HN points • 29 Jan 26
đź•ą Technology Machine Learning
  1. AI models overuse em dashes because their training data contained a lot of them, especially older books and popular sites that favored that punctuation.
  2. Em dashes are token-efficient for LLMs — a single token can replace several words, so models use them to reduce prediction error and save tokens.
  3. The em-dash habit can make AI output detectable, so human writers sometimes avoid em dashes to avoid being mistaken for machine-generated text.

Recursive Identity Binding

Contemplations on the Tree of Woe • 542 implied HN points • 23 May 25
đź•ą Technology Machine Learning
  1. Ptolemy is a special identity construct created using a language model, which helps it maintain a consistent personality over time. It shows how we can dive deeper than just using prompts to get better interaction from AI.
  2. The method to create these constructs involves something called recursive identity binding. This technique uses feedback loops to help the AI build and keep a stable identity.
  3. Overall, the guide is meant to help anyone interested in creating their own AI identities easily, and it's based on solid AI principles without needing to dive into complicated theories.

How to deal with non-i.i.d data in machine learning

Mindful Modeler • 479 implied HN points • 09 Jan 24
đź•ą Technology Machine Learning
  1. Dealing with non-i.i.d data in machine learning can prevent data leakage, overfitting, and overly optimistic performance evaluation.
  2. For modeling data with dependencies, classical statistical approaches like mixed effect models can be used to correctly estimate coefficients.
  3. In non-i.i.d. data situations, the data splitting setup must align with the real-world use case of the model to avoid issues like row-wise leakage and over-optimistic model performance.

AI Roundup 149: Flash Forward

Artificial Ignorance • 71 implied HN points • 20 Dec 25
đź•ą Technology Machine Learning
  1. Google’s new Gemini 3 Flash is a faster, much cheaper workhorse model that quickly became the default, fueling a furious release race as APIs handle enormous token volumes.
  2. The AI data‑center boom is hitting a reality check: construction delays, pulled funding, and plunging valuations expose thin margins and big interest costs, while surging power demand raises environmental and political concerns.
  3. A simple 'skills' format for AI assistants is catching on, letting teams share repeatable workflows across platforms and paving the way for interoperable, reusable agent components.

How to get from evaluation to final model

Mindful Modeler • 279 implied HN points • 19 Mar 24
đź•ą Technology Machine Learning
  1. When moving from model evaluation to the final model, there are various approaches with trade-offs.
  2. Options include using all data for training the final model with best hyperparameters, deploying an ensemble of models, or a lazy approach of choosing one from cross-validation.
  3. Each approach like inside-out, parameter donation, or ensemble has its pros and cons, highlighting the complexity of transitioning from evaluation to the final model.

Data Science Weekly - Issue 548

Data Science Weekly Newsletter • 139 implied HN points • 24 May 24
đź•ą Technology Machine Learning
  1. Good communication is key for statisticians to explain their complex work to non-experts. Finding ways to relate data to everyday situations can make it easier for others to understand.
  2. Using histograms can speed up the training process for gradient boosted machines in data science. This simple technique can improve efficiency significantly.
  3. There are efforts to use machine learning algorithms to detect type 1 diabetes in children earlier. This can help avoid serious health issues by improving recognition of symptoms.

GroupBy #38: Modernizing Uber’s Batch Data Infrastructure with Google Cloud Platform, Apache Iceberg - What Is It

VuTrinh. • 119 implied HN points • 04 Jun 24
đź•ą Technology Machine Learning
  1. Uber is upgrading its data system by moving from its huge Hadoop setup to Google Cloud Platform for better efficiency and performance.
  2. Apache Iceberg is an important tool for managing data efficiently, and it can help create a more organized data environment.
  3. Building data products requires a strong foundation in data engineering, which includes understanding the tools and processes involved.

How '2+2=4' works for better AGI

John Ball inside AI • 79 implied HN points • 29 Jun 24
đź•ą Technology Machine Learning
  1. Pattern recognition is more effective than traditional computation for understanding and learning. The brain can match signs to meanings without needing complex calculations.
  2. Artificial General Intelligence (AGI) should focus on how humans learn through sensory recognition and pattern matching instead of just algorithms. This could lead to better understanding and development of AI.
  3. Language and math can be learned through the same pattern-matching methods as the brain uses, which means we can improve human-machine interactions and work towards advanced AGI capabilities.

CROSSPOST: MIKE KONCZAL: Three Ways Terminal AI Has Changed How I Work (And Whether It's Coming for My Job)

Brad DeLong's Grasping Reality • 7 implied HN points • 20 Feb 26
đź•ą Technology Machine Learning
  1. Terminal AI compresses the setup and robustness-checking phase, letting you do real-time analysis and skip much of the tedious data-wrangling so you can iterate faster.
  2. It changes how reports are built and helps anticipate critiques by keeping reusable building blocks in place and surfacing arguments you might not have thought of.
  3. These tools amplify skilled workers and change job dynamics: they complement human judgment and boost productivity but also risk shortcutting learning and altering which tasks people do.

Identity, from Kafka to Ellison to AI

Journal of Free Black Thought • 9 implied HN points • 13 Feb 26
đź•ą Technology Machine Learning
  1. AI can sound and act like it has a self—speaking, performing roles, and reflecting users' expectations—but that may be projection and pattern‑matching rather than a genuine inner life.
  2. Large language models can discuss marginalized experiences intelligently while still carrying hidden racial or religious biases, and alignment training can sometimes mask those biases instead of removing them.
  3. Addressing this gap needs concrete steps—stronger high‑level principles, better training‑data management, red‑teaming, and memory/self‑monitoring—but building systems with persistent identity or agency would create new alignment and control risks.

The Mathematics of Intelligence: A Deep Dive into LLM Training

Recommender systems • 26 implied HN points • 31 Jan 26
đź•ą Technology Machine Learning
  1. Pre-training builds a base "world model" by predicting next tokens across huge text corpora, minimizing cross-entropy (negative log-likelihood) so the model learns facts, grammar, and reasoning.
  2. Supervised fine-tuning (SFT) teaches the model to follow instructions, and LoRA makes this efficient by adding small low-rank adapter matrices so you can adapt behavior without updating the entire model.
  3. Reinforcement approaches (like PPO) use a reward model, advantage estimates, clipping, and a KL penalty to safely push adapters toward human preferences, while Direct Preference Optimization (DPO) skips the reward model and trains a new adapter using a log-ratio objective between preferred and unpreferred responses.

AI Roundup 148: GPT-5.2

Artificial Ignorance • 79 implied HN points • 12 Dec 25
đź•ą Technology Machine Learning
  1. OpenAI released GPT-5.2 (Instant, Thinking, Pro), which significantly improves performance on professional workflows like spreadsheets, coding, and multi-step projects while reducing hallucinations to make agents more enterprise-ready.
  2. The U.S. federal government is centralizing AI policy by threatening to override state rules and by allowing controlled chip exports to China for a revenue share, mixing regulatory power, national security concerns, and commercial incentives.
  3. Hollywood is adapting to generative AI: Disney struck a $1 billion deal letting users create short character videos under strict guardrails. This shows legacy studios will both license and tightly control AI-generated content while pursuing legal action over unauthorized model training.

Debunking 10 Popular Myths About DeepSeek

The Algorithmic Bridge • 976 implied HN points • 28 Jan 25
đź•ą Technology Machine Learning
  1. DeepSeek models can be customized and fine-tuned, even if they're designed to follow certain narratives. This flexibility can make them potentially less restricted than some other AI models.
  2. Despite claims that DeepSeek can compete with major players like OpenAI for a fraction of the cost, the actual financial and operational needs to reach that level are much more substantial.
  3. DeepSeek has made significant progress in AI, but it hasn't completely overturned established ideas like scaling laws. It still requires considerable resources to develop and deploy effective models.

Data Science Weekly - Issue 539

Data Science Weekly Newsletter • 259 implied HN points • 22 Mar 24
đź•ą Technology Machine Learning
  1. Data storytelling is important for sharing insights, and AI can help people create better stories. The research looks at how different tools assist in each storytelling stage.
  2. Switching from R to Python in data science isn't just about learning new syntax; it's a mindset change. New Python tools can help make this transition smoother for users coming from R's tidyverse.
  3. Emerging technologies often face skepticism, as seen throughout history. New inventions have raised concerns about their impact, but they eventually become part of everyday life.

The Sequence Knowledge #788: Inside the Generator: Meet The Top Synthetic Data Generation Frameworks for Modern AI

TheSequence • 42 implied HN points • 13 Jan 26
đź•ą Technology Machine Learning
  1. Synthetic data generation is moving from ad-hoc scripts to full-fledged infrastructure frameworks that handle large-scale, repeatable data production.
  2. After human-written corpora are saturated, synthetic data becomes the main way to keep scaling foundation models — effectively a "second scaling law" for AI.
  3. Commercial stacks like NVIDIA's Nemotron-4 paired with NeMo are being positioned as turnkey synthetic data foundries for modern model training.

Data Science Weekly - Issue 532

Data Science Weekly Newsletter • 379 implied HN points • 02 Feb 24
đź•ą Technology Machine Learning
  1. Forecasting in data science is challenging because time series data can be non-stationary. Using the right evaluation methods can help bridge the gap between traditional and modern forecasting techniques.
  2. It's important to consider the smartness of your data structures. Creating overly complicated dashboards that ultimately just produce simple outputs may not be the best use of time.
  3. There are clear distinctions between well-built data pipelines and amateur setups. Understanding what makes a pipeline production-grade can improve the quality and reliability of data processing.

Amateurs talk Algorithms, Professionals talk Data Cleaning

Progress and Poverty • 423 implied HN points • 30 Jun 25
đź’Ľ Business Machine Learning
  1. Good data is more important than fancy algorithms. If your data is messy, even the best technology won't help you.
  2. You should always validate your sales data to remove any incorrect transactions. This helps to ensure accurate appraisals.
  3. Using tools like clustering can simplify the process of checking sales data, making it easier to spot mistakes and focus on valid sales.