The hottest Natural Language Substack posts right now

And their main takeaways
Category
Top Technology Topics

Bits In, Bits Out

The Intrinsic Perspective • 43156 implied HN points • 05 Mar 26
🕹 Technology Natural Language
  1. LLMs are tools that boost efficiency and scale but mostly imitate human input; without detailed prompts and human scaffolding they produce shallow, imitative output.
  2. Instead of a sudden intelligence explosion, LLMs have contributed a glut of mediocre text—average book quality dropped while the very best works changed little.
  3. That pattern will likely spread to other fields like science and math: skilled users get modest gains, the world is buried in low-quality output, and human expertise remains essential rather than being replaced by autonomous superintelligence.

The Weekly Kaitchup #65

The Kaitchup – AI on a Budget • 59 implied HN points • 01 Nov 24
🕹 Technology Natural Language
  1. SmolLM2 offers alternatives to popular models like Qwen2.5 and Llama 3.2, showing good performance with various versions available.
  2. The Layer Skip method improves the speed and efficiency of Llama models by processing some layers selectively, making them faster without losing accuracy.
  3. MaskGCT is a new text-to-speech model that generates high-quality speech without needing text alignment, providing better results across different benchmarks.

GPT-5.4 Is A Substantial Upgrade

Don't Worry About the Vase • 3270 implied HN points • 11 Mar 26
🕹 Technology Natural Language
  1. GPT-5.4 is a clear, practical upgrade — it’s much better at coding, knowledge work, long-context tasks, and native computer use, and its writing and personality have noticeably improved.
  2. Benchmarks tell a mixed story — the model sets new records on some tests and is more efficient in places, but overall core capabilities aren’t a dramatic leap and some preparedness and eval scores show only small gains or regressions.
  3. Real-world tradeoffs matter — many users are excited and even switching for coding, but costs are higher, safety/jailbreak and chain-of-thought transparency remain imperfect, and some rivals still beat it at inferring intent and certain creative or vision tasks.

BREAKING: LLM “reasoning” continues to be deeply flawed

Marcus on AI • 15216 implied HN points • 10 Feb 26
🕹 Technology Natural Language
  1. Large language models still routinely make reasoning mistakes and hallucinate, so they are not reliable for true logical or causal reasoning.
  2. A broad, careful review found these failures are widespread across recent models, showing that massive funding and scaling alone haven’t solved reasoning.
  3. The field faces a choice: keep dismissing critics and double down on scale, or acknowledge the limits and invest in alternative approaches that directly address reasoning.

Breaking: Sir Demis Hassabis becomes the latest to say that ChatGPT is a dead-end and that we must turn our focus to world models

Marcus on AI • 9366 implied HN points • 22 Jan 26
🕹 Technology Natural Language
  1. A leading AI figure says ChatGPT-style large language models are a dead end and researchers should prioritize building world models.
  2. This comment joins other voices pushing the field to move beyond chat interfaces toward systems that actually model and understand the world.
  3. Earlier analysis argues that purely statistical approaches have limits and that neurosymbolic or cognitive 'world' models are needed for deeper AI.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Voyaging on Strange Seas of Thought, Not, However, Alone!: Where I Am Finding MAMLMs Useful This Winter

Brad DeLong's Grasping Reality • 115 implied HN points • 23 Feb 26
🕹 Technology Natural Language
  1. Treat modern advanced language models as token‑producing tools and database interfaces, not as minds, friends, or co‑authors.
  2. The key skill is context engineering and attention management: carefully fill the context window, use external scratchpads or state, select and compress relevant material, and isolate tasks to avoid interference.
  3. Build reliable tool‑based workflows — copilots, constrained formats, verification loops, and domain evaluators — to filter, summarize, and connect you to collective human knowledge instead of treating the model as the source of wisdom.

Deep Learning Legends: Ilya's List, or 90% of Everything That Matters in AI

Gonzo ML • 252 implied HN points • 08 Feb 26
🕹 Technology Natural Language
  1. A compact, curated reading list of landmark papers can teach roughly 90% of the core ideas and techniques in deep learning, offering a fast path to real understanding.
  2. The essential topics span sequence models (RNNs/LSTMs/NTM), attention and transformers, convolutional vision models, theory of complexity and description length, training methods and scaling, and multimodal/speech work.
  3. The publicly available partial list misses several important areas — notably reinforcement learning and meta-learning — so it should be supplemented with RL classics and recent advances like scaling laws, compute‑optimal training, mixture‑of‑experts, distillation, and key optimization tricks.

The Sequence AI of the Week #801: Deconstructing Kimi 2.5

TheSequence • 245 implied HN points • 04 Feb 26
🕹 Technology Natural Language
  1. Kimi 2.5 represents a paradigm shift from scale-driven "emergence" to orchestration, where the model coordinates complex workflows instead of just generating text.
  2. It functions as an end-to-end agent that manages execution environments, spawns subprocesses, and debugs its own visual outputs in a closed-loop system.
  3. The system uses sparsity to deliver trillion-parameter capability with the latency and cost profile similar to a ~32B dense model.

AI links, 1/18/2026

In My Tribe • 243 implied HN points • 18 Jan 26
🕹 Technology Natural Language
  1. Many state AI bills will be written as chatbot rules and will miss coding agents, so policy risk becoming outdated very quickly.
  2. Advanced coding agents like Claude Code with Opus 4.5 are producing big productivity gains and could change how people interact with computers beyond simple Q&A chatbots.
  3. LLMs are largely backward-looking and poor at spotting fast-moving trends, and while AI can make professions like law more efficient it can also reduce billable hours and create confidentiality risks if client data is used for training.

LLMs – Part 3: Context Matters — Self Attention

Vasu’s Newsletter • 78 implied HN points • 25 Jan 26
🕹 Technology Natural Language
  1. Each token creates query, key, and value vectors so it can ask what it needs, match that against other tokens, and gather useful information.
  2. Tokens compare their query to every key to get raw scores, convert those scores to attention weights with softmax, and use the weights to take a weighted sum of value vectors to produce a new contextual vector.
  3. Self-attention makes token meanings contextual (helping with pronouns, disambiguation, and long-range links), and models use multiple attention heads plus feed-forward layers to capture different relation patterns and refine each token's representation.

Don’t Ride This Bike! Generative AI’s persistent trouble with compositionality and parts

Marcus on AI • 3952 implied HN points • 08 Dec 24
🕹 Technology Natural Language
  1. Generative AI struggles with understanding complex relationships between objects in images. It sometimes produces physically impossible results or gets details wrong when asked to create images from text.
  2. Recent improvements in AI models, like DALL-E3, show only slight progress in handling specifications related to parts of objects. It can still mislabel parts or fail to follow more complex requests.
  3. AI systems need to improve their ability to check and confirm that generated images match the prompts given by users. This may require new technologies for better understanding between language and visuals.

Top 10 Books we read in 2025

Graphs For Science • 105 implied HN points • 10 Jan 26
🕹 Technology Natural Language
  1. A strong theme is practical engineering: many books show how to turn LLM demos into working agents using RAG, embeddings, knowledge graphs, tool use, and prompt patterns to make outputs more reliable and auditable.
  2. There’s a clear focus on hands-on playbooks and trade-offs—quick-starts, checklists, code examples, and patterns for prototyping, retrieval, latency/cost decisions, multi-agent orchestration, and production concerns.
  3. The collection balances technical how-to guidance with broader perspectives on responsible use, human uniqueness, organizational strategy, and interdisciplinary science, highlighting ethics, norms for academics, and big-picture questions about life and intelligence.

AI Agents: Exploring Agentic Applications

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 119 implied HN points • 29 Jul 24
🕹 Technology Natural Language
  1. Agentic applications are AI systems that can perform tasks and make decisions on their own, using advanced models. They can adapt their actions based on user input and the environment.
  2. OpenAgents is a platform designed to help regular users interact with AI agents easily. It includes different types of agents for data analysis, web browsing, and integrating daily tools.
  3. For these AI agents to work well, they need to be user-friendly, quick, and handle mistakes gracefully. This is important to ensure that everyone can use them, not just tech experts.

Using GPT LLM MAMLMs as Your Rabbit—Your Pacer

Brad DeLong's Grasping Reality • 261 implied HN points • 22 Nov 25
🕹 Technology Natural Language
  1. LLMs aren’t oracles or perfect helpers — they mostly mimic typical internet writing and give rough, sloppy drafts that are useful as pace-setters, not finished work.
  2. All the tricks to make them better (context engineering, fine-tuning, RAG, etc.) are heavy, fragile, and costly patches. Only invest in that work when you really need high-volume or specialized, production-ready output.
  3. AI can lift weak writers and handle boilerplate well, but for persuasive or high-quality writing the best workflow is to use the model for a rough draft and then heavily rewrite it into something authentic.

Wetware-Hardware Centaurs, Not Digital Gods: Wednesday MAMLMs

Brad DeLong's Grasping Reality • 176 implied HN points • 26 Nov 25
🕹 Technology Natural Language
  1. Modern large language models are super-fast next-token mimics that draw on the collective human text record but don’t have durable world models, so they can be very good at summarizing and pattern-matching yet fail at understanding time, causality, or embodied tasks.
  2. AI capabilities are jagged: models shine on problems with clear reward signals or when the needed context fits easily into their input window, but they fail unpredictably on other practical tasks, and raw hardware speed alone won’t erase that unevenness.
  3. The realistic near-term outcome is centaur workflows where humans provide judgment and guardrails; achieving true, general understanding likely requires rethinking architectures to build explicit world models rather than just scaling current next-token engines.

The Sequence Knowledge #792: EVERYTHING you Need to Know About Synthetic Data Generation

TheSequence • 49 implied HN points • 20 Jan 26
🕹 Technology Natural Language
  1. Synthetic data is a practical scaling lever that fills coverage gaps and builds long-tail capabilities by creating targeted examples instead of waiting for rare real-world labels.
  2. Core methods include generative synthesis, rephrasing/paraphrasing, multi-turn dialogue synthesis, and RL trajectory generation, each tailored to different tasks like images, instructions, conversations, or environment rollouts.
  3. The focus is on quality over quantity: tight specs, automatic verification, diversity controls, and eval-driven feedback let teams steer capabilities, improve class balance, protect privacy, and iterate quickly.

The Sequence AI of the Week #789: Recursive Language Models: Inside the MIT Research Everyone is Talking About

TheSequence • 56 implied HN points • 14 Jan 26
🕹 Technology Natural Language
  1. Bigger context windows aren't always the answer; dumping more text into attention can make a model's reasoning worse, not better.
  2. The paper calls this failure mode "context rot": as prompts grow, attention dilutes, the model's working set becomes unmanageable, and output quality drops.
  3. Instead of just expanding attention, we need different computational shapes—treating prompts more like environments and processing information recursively to avoid drowning the model in irrelevant context.

The Mathematics of Intelligence: A Deep Dive into LLM Training

Recommender systems • 26 implied HN points • 31 Jan 26
🕹 Technology Natural Language
  1. Pre-training builds a base "world model" by predicting next tokens across huge text corpora, minimizing cross-entropy (negative log-likelihood) so the model learns facts, grammar, and reasoning.
  2. Supervised fine-tuning (SFT) teaches the model to follow instructions, and LoRA makes this efficient by adding small low-rank adapter matrices so you can adapt behavior without updating the entire model.
  3. Reinforcement approaches (like PPO) use a reward model, advantage estimates, clipping, and a KL penalty to safely push adapters toward human preferences, while Direct Preference Optimization (DPO) skips the reward model and trains a new adapter using a log-ratio objective between preferred and unpreferred responses.

AGI won't use computations

John Ball inside AI • 79 implied HN points • 23 Jun 24
🕹 Technology Natural Language
  1. Artificial General Intelligence (AGI) might be achieved by focusing on pattern matching rather than traditional computations. This means understanding and recognizing complex patterns, just like how our brains work.
  2. Current AI systems struggle with tasks like driving or conversing naturally because they don't operate like human brains. Instead of tightly-coupled algorithms, more flexible and efficient pattern-based systems might be the key.
  3. Patom theory suggests that brains store and match patterns in a unique way, which allows for better learning and error correction. By applying these ideas, we could improve AI systems to be more human-like in understanding and interaction.

A Vector Database 2,000 years in the making

What Is Called Thinking? • 82 implied HN points • 25 Nov 25
🕹 Technology Natural Language
  1. The Oral Torah is described as a living, growing, self-referential commentary tradition that developed over two thousand years and across continents.
  2. It’s not just an “oral tradition” that was later written down, but an ongoing, networked conversation of interpretation and commentary.
  3. The piece asks whether people should write with AIs in mind and suggests imagining the Oral Torah as a kind of long-lived, interconnected repository—like a vector database—for modern LLMs.

Da fuq is an “LLM?”

Sex and the State • 26 implied HN points • 14 Jan 26
🕹 Technology Natural Language
  1. An LLM (large language model) is an AI system that mainly reads and writes natural language and powers modern chatbots like ChatGPT, Claude, and Gemini.
  2. AI is a big umbrella with many types of tools — image generators, detectors, chat interfaces, and world models — and LLMs are just the language-focused slice, not the same as models that work with images or spatial data.
  3. Many leading researchers argue LLMs alone probably won’t produce human-level or general intelligence, because language only points to thought; building AGI likely requires spatial or "world" models that learn from videos, perception, and interaction.

Languages don't need many words

John Ball inside AI • 39 implied HN points • 24 Jul 24
🕹 Technology Natural Language
  1. You don't need many words to communicate in a new language. Just a small vocabulary can help you get by in everyday conversations.
  2. For understanding most spoken and written text, around 2000 words are usually enough. This covers about 80% of regular communication.
  3. Machine learning and AI can benefit from understanding language like humans do, by learning new words in context rather than just relying on a large vocabulary.

o3: The grand finale of AI in 2024

Democratizing Automation • 815 implied HN points • 20 Dec 24
🕹 Technology Natural Language
  1. OpenAI's new model, o3, is a significant improvement in AI reasoning. It will be available to the public in early 2025, and many experts believe it could change how we use AI.
  2. The o3 model has shown it can solve complex tasks better than previous models. This includes performing well on math and coding benchmarks, marking a big step for AI.
  3. As the costs of using AI decrease, we can expect to see these models used more widely, impacting jobs and industries in ways we might not yet fully understand.

Teaching Small Language Models to Reason

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 39 implied HN points • 10 Jul 24
🕹 Technology Natural Language
  1. Using Chain-Of-Thought prompting helps large language models think through problems step by step, which makes them more accurate in their answers.
  2. Smaller language models struggle with Chain-Of-Thought prompting and often get confused because they don't have enough knowledge and understanding like the bigger models.
  3. Google Research has a method to teach smaller models by learning from larger ones. This involves using the bigger models to create helpful examples that the smaller models can then learn from.

RAG Survey & Available Research

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 39 implied HN points • 27 Jun 24
🕹 Technology Natural Language
  1. Retrieval-Augmented Generation (RAG) mixes retrieval methods with learning systems to help large language models use real-time data.
  2. RAG can enhance the accuracy of language models by incorporating current information, avoiding wrong answers that might come from outdated knowledge.
  3. The framework of RAG includes steps like pre-retrieval, retrieval, post-retrieval, and generation, each contributing to better outputs in language processing tasks.

TinyStories

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 39 implied HN points • 26 Jun 24
🕹 Technology Natural Language
  1. Phi-3 is a small language model that uses a special dataset called TinyStories. This dataset was designed to help the model create more varied and engaging stories.
  2. TinyStories uses simple vocabulary suitable for young children, focusing on quality over quantity. The stories generated are meant to be both understandable and entertaining.
  3. Training the Phi-3 model with TinyStories can be done quickly and allows for easier fine-tuning. This helps smaller organizations use advanced language models without needing huge resources.

Inside AI

John Ball inside AI • 39 implied HN points • 12 Jun 24
🕹 Technology Natural Language
  1. AGI might not come from current machine learning methods. Instead, understanding how human brains work could be the key to achieving it.
  2. The theory behind brain functions can help solve AI challenges. Learning from how brains process information could lead us to better AI solutions.
  3. Language is crucial for interacting with AI. Building a trustworthy AI community focused on language can improve how we communicate and use technology.

Mondays with the Machine: The Tongue & the Token: Language as Interface in Our Current Age of AI

Brad DeLong's Grasping Reality • 169 implied HN points • 09 Jun 25
🕹 Technology Natural Language
  1. Natural language interfaces are a big deal because they let us communicate with AI using everyday language. This makes it easier for everyone to use technology without needing to know complex coding or technical skills.
  2. AI systems, like language models, simulate understanding but don't actually think. They can help us find information and assist with tasks, but we should remember that they are not truly intelligent.
  3. Using conversational AI can democratize access to information, making it easier for people to learn and solve problems. However, we must be aware of the risks, like over-reliance on these systems.

Moving From Natural Language Understanding To Mobile UI Understanding

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 19 implied HN points • 11 Jul 24
🕹 Technology Natural Language
  1. Natural Language Understanding (NLU) helps machines grasp and respond to human language, making sense of unstructured conversations.
  2. The shift to Mobile UI Understanding means we are now focused on understanding what's on mobile screens instead of just conversations.
  3. The Ferret-UI model enables devices to interact with users in a more meaningful way, allowing for richer and more context-aware conversations.

RAG, Hallucination & Structure: Research By ServiceNow

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 59 implied HN points • 18 Apr 24
🕹 Technology Natural Language
  1. ServiceNow is using a method called Retrieval-Augmented Generation (RAG) to help transform user requests in natural language into structured workflows. This aims to improve how easily users can create workflows without needing deep technical knowledge.
  2. By using RAG, they want to reduce 'hallucination', which is when AI generates wrong or irrelevant info, and make the AI more reliable. This is important for gaining user trust in AI systems.
  3. The study also suggests future improvements, like changing output formats for efficiency and streamlining processes so that users can see steps one at a time, making it easier to follow along.

Accio Insights: The Marauder’s Map of the ML World

The Palindrome • 3 implied HN points • 19 Feb 26
🕹 Technology Natural Language
  1. Embeddings are learned, dense numerical vectors that capture what words or items mean in context instead of using one‑hot or random encodings.
  2. Similarity in embedding space is measured by the cosine of the angle between vectors, and relationships show up as directions you can add or subtract (for example, king − man + woman ≈ queen), so similar things cluster and outliers stand out.
  3. Embeddings are a core building block across ML systems — powering search, LLMs, image generators, and recommendations — and engineers must design around retrieval, scale, latency, and reliability when using them in production.

LLM Disruption in Chatbot Development Frameworks

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 19 implied HN points • 05 Jul 24
🕹 Technology Natural Language
  1. Large Language Models (LLMs) make chatbots act more like humans, making it easier for developers to create smart bots.
  2. Using LLMs reduces the need for complex programming rules, allowing for quicker chatbot setup for different uses.
  3. Despite the benefits, there are still challenges, like keeping chatbots stable and predictable as they become more advanced.

Improve Conversational UIs Using Social Intelligence

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 59 implied HN points • 09 Apr 24
🕹 Technology Natural Language
  1. Social intelligence is important for conversational AIs to feel more human-like. It helps them understand emotions and social cues better.
  2. A good conversational UI needs to consider cognitive, situational, and behavioral intelligence. This means the AI should know what you mean, the context of your words, and how to interact appropriately.
  3. Using more data and different types of information beyond just words can help improve how AIs communicate. This could include things like images and gestures to understand conversations better.

A Short History Of Chatbots

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 39 implied HN points • 09 May 24
🕹 Technology Natural Language
  1. Chatbots have changed a lot over time, starting as simple rule-based systems and moving to advanced AI models that can understand context and user intent.
  2. Early chatbots used basic pattern recognition to respond to user questions, but this method was limited and often resulted in repetitive and predictable answers.
  3. Now, modern chatbots utilize natural language understanding and machine learning to provide more dynamic and relevant responses, making them better at handling various conversations.

Can Conversation Designers Excel As Data Designers?

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 19 implied HN points • 24 Jun 24
🕹 Technology Natural Language
  1. Conversation designers can play a key role in creating and improving datasets for training language models. Their skills can help make data more relevant and useful.
  2. Techniques like Partial Answer Masking and Prompt Erasure help models learn to self-correct and think strategically. This makes them better at reasoning and understanding complex tasks.
  3. Chain-of-Thought methods help language models break down problems into smaller steps. This approach can lead to more accurate and reliable answers.

English is NOT a Programming Language 🤦

Sector 6 | The Newsletter of AIM • 79 implied HN points • 07 Feb 24
🕹 Technology Natural Language
  1. English has too many ambiguities to be a programming language. Programming needs precise rules, and English doesn't always follow them.
  2. Douglas Crockford, the creator of JSON, is worried about pushing English as a coding language. He believes that code must be perfect, which English is not.
  3. Using natural language through AI for programming might lead to confusion. Clarity and accuracy are crucial for writing successful code.

Large Impact: The Rise of Small Language Models

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 59 implied HN points • 07 Mar 24
🕹 Technology Natural Language
  1. Small Language Models (SLMs) are becoming popular because they are easier to access and can run offline. This makes them appealing to more users and businesses.
  2. While Large Language Models (LLMs) are powerful, they can give wrong answers or lack up-to-date information. SLMs can solve many problems without these issues.
  3. Using Retrieval-Augmented Generation (RAG) with SLMs can help them answer questions better by providing the right context without needing extensive knowledge.

Data Science Weekly - Issue 495

Data Science Weekly Newsletter • 239 implied HN points • 19 May 23
🕹 Technology Natural Language
  1. Absence of evidence can often serve as strong evidence of absence, and this idea can be explored with Bayesian methods.
  2. Natural language processing is being used to analyze global supply chains, helping create networks from news articles.
  3. It's crucial to understand the unique challenges and opportunities in personalizing search results, as seen with Netflix's approach.

Implementing Chain-of-Thought Principles in Fine-Tuning Data for RAG Systems

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 19 implied HN points • 07 Jun 24
🕹 Technology Natural Language
  1. Using Chain-of-Thought principles can help language models improve how they think and respond. This means they can become better at understanding complex questions.
  2. Fine-tuning training data is being done in a more detailed way to enhance performance. This makes the models more efficient and effective in answering specific tasks.
  3. The goal of these improvements is to reduce errors, or 'hallucinations,' in responses. This way, the model can provide more accurate answers based on the information it retrieves.

Chain-of-Instructions (CoI) Fine-Tuning & Going Beyond Instruction Tuning

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 39 implied HN points • 21 Mar 24
🕹 Technology Natural Language
  1. Chain-of-Instructions (CoI) fine-tuning allows models to handle complex tasks by breaking them down into manageable steps. This means that a task can be solved one part at a time, making it easier to follow.
  2. This new approach improves the model's ability to understand and complete instructions it hasn't encountered before. It's like teaching a student to tackle complex problems by showing them how to approach each smaller task.
  3. Training with minimal human supervision leads to efficient dataset creation that can empower models to reason better. It's as if the model learns on its own, becoming smarter and more capable through well-designed training.