The hottest Model Scaling Substack posts right now

And their main takeaways
Category
Top Technology Topics

Deep Learning Legends: Ilya's List, or 90% of Everything That Matters in AI

Gonzo ML • 252 implied HN points • 08 Feb 26
đź•ą Technology Model Scaling
  1. A compact, curated reading list of landmark papers can teach roughly 90% of the core ideas and techniques in deep learning, offering a fast path to real understanding.
  2. The essential topics span sequence models (RNNs/LSTMs/NTM), attention and transformers, convolutional vision models, theory of complexity and description length, training methods and scaling, and multimodal/speech work.
  3. The publicly available partial list misses several important areas — notably reinforcement learning and meta-learning — so it should be supplemented with RL classics and recent advances like scaling laws, compute‑optimal training, mixture‑of‑experts, distillation, and key optimization tricks.

Is AI progress slowing down?

AI Snake Oil • 1297 implied HN points • 18 Dec 24
đź•ą Technology Model Scaling
  1. The idea that AI progress is surely slowing down might be too hasty. We may not have explored all the ways to improve AI through model scaling just yet.
  2. Industry experts often change their predictions about AI, showing that they might not know as much as we assume. Their interests can influence their views, so take their forecasts with a grain of salt.
  3. While new methods like inference scaling can boost AI capabilities quickly, the actual impact on real-world applications may take time due to product development lags and varying reliability.

The Sequence AI of the Week #785: Gradient Highway Maintenance: Inside DeepSeek’s Latest Breakthrough

TheSequence • 35 implied HN points • 07 Jan 26
đź•ą Technology Model Scaling
  1. DeepSeek's mHC challenges established assumptions about AI scaling and suggests new architectural ideas that could change how larger models are built and trained.
  2. Residual connections are the unsung scaffolding of modern deep networks, providing a 'gradient highway' that keeps training stable across many layers.
  3. The simple rule y = f(x) + x—adding the input back to a layer's output—was revolutionary because it preserves signals and gradients, making very deep networks trainable.

TheSequence Opinion #699: 2030 or Bust? The Compute Surge and the Bottlenecks Ahead

TheSequence • 84 implied HN points • 07 Aug 25
đź•ą Technology Model Scaling
  1. Artificial General Intelligence (AGI) might be possible by 2030 if we keep improving our computing power and models.
  2. However, there are worries that after 2030, we could hit limits with our technology that will require us to find new ways to innovate.
  3. We might need better algorithms and improved designs because just making computers bigger and faster won't be enough forever.

AI Foundations Part 1: Transformers, Pre-Training and Fine-Tuning, and Scaling

Mule’s Musings • 378 implied HN points • 11 Apr 23
đź•ą Technology Model Scaling
  1. The Transformer model revolutionized Large Language Models (LLMs) with its parallel and scalable architecture.
  2. Pre-training and fine-tuning, as seen in GPT-1 and BERT, significantly improved model performance for various tasks.
  3. Bigger models, more data, and computing power have shown to lead to better performance in LLMs, but the relationship between model size, training tokens, and performance is more complex than initially thought.
Get a weekly roundup of the best Substack posts, by hacker news affinity: