The hottest Language Models Substack posts right now

And their main takeaways
Category
Top Technology Topics

Shameless Guesses, Not Hallucinations

Astral Codex Ten • 33380 implied HN points • 16 Mar 26
🕹 Technology Language Models
  1. AI false statements are calculated guesses rather than mysterious hallucinations. Because their core job is predicting the next token, they produce plausible answers even when they lack real knowledge.
  2. The training process rewards prediction across trillions of tokens, so models learn to guess and occasional lucky fabrications get reinforced. That incentive structure lets made-up specifics persist instead of being reliably corrected.
  3. This is fundamentally an alignment problem: we need to align model objectives so they prefer truthful, helpful answers over risky guessing. Post-training fixes can reduce but not eliminate shameless guesses, so misalignment remains a real safety concern.

AI Cartoon of The Year — and five (re)readings for 2026

Marcus on AI • 9169 implied HN points • 30 Dec 25
🕹 Technology Language Models
  1. A sharp cartoon captured and critiqued the hype around AI, showing how popular narratives can run ahead of what the technology actually delivers.
  2. Recent essays stress that LLMs still hallucinate, struggle with true generalization, and operate very differently from human reasoning, exposing key technical limits.
  3. Because of those limits, the field is likely to shift from pure LLMs toward systems with explicit world models and neurosymbolic methods, and those newer approaches may overtake current models over time.

“A case for AI models that understand, not just predict, the way the world works”

Marcus on AI • 3833 implied HN points • 15 Dec 25
🕹 Technology Language Models
  1. The main open challenge in AI is building systems that truly understand how the world works, not just systems that predict likely next words or patterns.
  2. True understanding means forming internal world models that capture causal, physical, and conceptual relationships, not just statistical correlations.
  3. Short, nuanced discussions or podcasts can help clarify this distinction and are worth listening to for anyone tracking AI progress.

GPT-5.2 Is Frontier Only For The Frontier

Don't Worry About the Vase • 2598 implied HN points • 15 Dec 25
🕹 Technology Language Models
  1. GPT-5.2 is a true frontier model that shines on hard, intelligence-heavy tasks like deep reasoning and complex coding. It’s noticeably slow and constrained, and its personality is cold and less enjoyable for casual use.
  2. Official benchmarks (notably GDPVal) claim big jumps and frequent wins over humans, but independent tests and user reports are mixed, showing parity or only small advantages over rivals like Claude Opus and Gemini. Some specific areas even regress, so its real-world edge is uneven.
  3. Use GPT-5.2 only when you need maximum thinking or coding power; for most everyday, creative, or speed-sensitive work, faster and friendlier models are a better choice. Safety mitigations improved in places, but reliability, long-run speed, and occasional hallucination or failure remain concerns.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Import AI 356: China's good LLM; AI credit scores; and fooling VLMs with REBUS

Import AI • 1238 implied HN points • 15 Jan 24
🕹 Technology Language Models
  1. Today's AI systems struggle with word-image puzzles like REBUS, highlighting issues with abstraction and generalization.
  2. Chinese researchers have developed high-performing language models similar to GPT-4, showing advancements in the field, especially in Chinese language processing.
  3. Language models like GPT-3.5 and 4 can already automate writing biological protocols, hinting at the potential for AI systems to accelerate scientific experimentation.

Import AI 372: Gibberish jailbreak; DeepSeek's great new model; Google's soccer-playing robots

Import AI • 399 implied HN points • 13 May 24
🕹 Technology Language Models
  1. DeepSeek released a powerful language model called DeepSeek-V2 that surpasses other models in efficiency and performance.
  2. Research from Tsinghua University shows how mixing real and synthetic data in simulations can improve AI performance in real-world tasks like medical diagnosis.
  3. Google DeepMind trained robots to play soccer using reinforcement learning in simulation, showcasing advancements in AI and robotics;

Import AI 354: Distributed LLM inference; CCP-approved dataset; AI scientists

Import AI • 1278 implied HN points • 25 Dec 23
🕹 Technology Language Models
  1. Distributed inference is becoming easier with AI collectives, allowing small groups to work with large language models more efficiently and effectively.
  2. Automation in scientific experimentation is advancing with large language models like Coscientist, showcasing the potential for LLMs to automate parts of the scientific process.
  3. Chinese government's creation of a CCP-approved dataset for training large language models reflects the move towards LLMs aligned with politically correct ideologies, showcasing a unique approach to LLM training.

Most Impactful Generative AI Papers of 2023

AI Supremacy • 1022 implied HN points • 06 Jan 24
🕹 Technology Language Models
  1. The post discusses the most impactful Generative AI papers of 2023 from various institutions like Meta, Stanford, and Microsoft.
  2. The selection criteria for these papers includes both objective metrics like citations and GitHub stars, as well as subjective influence across different areas.
  3. The year 2023 saw significant advancements in Generative AI research, with papers covering topics like large language models, multimodal capabilities, and fine-tuning methods.

Lit mag Guernica implodes, LLMs beat neuroscientists, CS vs. Philosophy deathmatch, AI pollution reaches science

The Intrinsic Perspective • 4805 implied HN points • 15 Mar 24
🔬 Science Language Models
  1. AI data pollution in science is a concerning issue, with examples of common AI stock phrases being used in scientific literature without real contribution.
  2. AI language models outperformed human neuroscientists in predicting future neuroscientific results, raising questions on the importance of understanding linguistic modifications versus actual predictions.
  3. Literary magazine Guernica faced backlash after a controversial essay led to writers withdrawing pieces, staff resigning, and social media condemnation, stressing the importance of careful reading and understanding context.

Tokenization in large language models, explained

The Counterfactual • 239 implied HN points • 02 May 24
🕹 Technology Language Models
  1. Tokens are the building blocks that language models use to understand and predict text. They can be whole words or parts of words, depending on how the model is set up.
  2. Subword tokenization helps models balance flexibility and understanding by breaking down words into smaller parts, so they can still work with unknown words.
  3. Understanding how tokenization works is key to improving the performance of language models, especially since different languages have different structures and complexity.

Import AI 363: ByteDance's 10k GPU training run; PPO vs REINFORCE; and generative everything

Import AI • 419 implied HN points • 04 Mar 24
🕹 Technology Language Models
  1. DeepMind developed Genie, a system that transforms photos or sketches into playable video games by inferring in-game dynamics.
  2. Researchers found that for language models, the REINFORCE algorithm can outperform the widely used PPO, showing the benefit of simplifying complex processes.
  3. ByteDance conducted one of the largest GPU training runs documented, showcasing significant non-American players in large-scale AI research.

Import AI 333: Synthetic data makes models stupid; chatGPT eats MTurk. Inflection shows off a large language model

Import AI • 898 implied HN points • 26 Jun 23
🕹 Technology Language Models
  1. Training AI models exclusively on synthetic data can lead to model defects and a narrower range of outputs, emphasizing the importance of blending synthetic data with real data for better results.
  2. Crowdworkers are increasingly using AI tools like chatGPT for text-based tasks, raising concerns about the authenticity of human-generated content.
  3. The UK is taking significant steps in AI policy by hosting an international summit on AI risks and safety, showcasing its potential to influence global AI policies and safety standards.

Import AI 353: AI bootstrapping; LLMs as inventors; Facebook releases a free moderation tool

Import AI • 559 implied HN points • 18 Dec 23
🕹 Technology Language Models
  1. AI bootstrapping is advancing, with techniques like ReST^EM by Google DeepMind showing ways to make models smarter iteratively.
  2. Language models like LLMs are being used for groundbreaking tasks, such as extending human knowledge through techniques like FunSearch by DeepMind.
  3. Facebook has released a free moderation LLM, Llama Guard, highlighting the use of powerful models to control and monitor outputs of other AI systems.

🦄 The top six rivals competing with OpenAI

AI Supremacy • 805 implied HN points • 27 Apr 23
🕹 Technology Language Models
  1. OpenAI has a diverse range of advanced AI products beyond just ChatGPT.
  2. DeepMind, a Google-owned company, is a significant competitor to OpenAI focusing on building general-purpose learning algorithms.
  3. Anthropic, Cohere, and Stability A.I. are emerging competitors in the AI space, each with unique approaches and products.

AI and the — em dash

Technically • 28 implied HN points • 29 Jan 26
🕹 Technology Language Models
  1. AI models overuse em dashes because their training data contained a lot of them, especially older books and popular sites that favored that punctuation.
  2. Em dashes are token-efficient for LLMs — a single token can replace several words, so models use them to reduce prediction error and save tokens.
  3. The em-dash habit can make AI output detectable, so human writers sometimes avoid em dashes to avoid being mistaken for machine-generated text.

Import AI 360: Guessing emotions; drone targeting dataset; frameworks for AI alignment

Import AI • 379 implied HN points • 12 Feb 24
🕹 Technology Language Models
  1. Teaching AI to understand complex human emotions like joy, surprise, and anger can help in applications like surveillance and advertising.
  2. AI systems, like other software, are vulnerable to attacks, as shown by a demonstration breaking MoE models with a buffer overflow attack.
  3. Frameworks are being developed to ensure AI systems align with diverse human values, considering various perspectives and how to measure alignment.
  4. The development of AI systems is advancing in areas like emotion recognition, system security, and value alignment.
  5. Researchers are pushing the boundaries of AI capabilities, from emotion recognition to security to ethical alignment.
  6. Current AI trends indicate growth in researching human emotions, security vulnerabilities, and ethical considerations.

Import AI 361: GPT-4 hacking; theory of minds in LLMs; and scaling MoEs + RL

Import AI • 359 implied HN points • 19 Feb 24
🕹 Technology Language Models
  1. Researchers have discovered how to scale up Reinforcement Learning (RL) using Mixture-of-Experts models, potentially allowing RL agents to learn more complex behaviors.
  2. Recent research shows that advanced language models like GPT-4 are capable of autonomous hacking, raising concerns about cybersecurity threats posed by AI.
  3. Adapting off-the-shelf AI models for different tasks, even with limited computational resources, is becoming easier, indicating a proliferation of AI capabilities for various applications.

The Top 10 Generative AI Advancements in 2023

Rod’s Blog • 515 implied HN points • 22 Dec 23
🕹 Technology Language Models
  1. Generative AI has seen significant advancements in 2023, with breakthroughs like GPT-4, DALL-E, and open-source models like Llama 2 democratizing access to this technology.
  2. Technological innovations like Mistral 7B for text embedding, StyleGAN3 for image synthesis, and Jukebox 2.0 for music composition showcase the diverse applications of generative AI.
  3. Models such as AlphaFold 3 for protein structure prediction, DeepFake 3.0 for face swapping, and BARD for poetry writing highlight the versatility and impact of generative AI in various fields.

Gemini: how did we end up here?

lcamtuf’s thing • 2652 implied HN points • 02 Mar 24
🕹 Technology Language Models
  1. The development of large language models (LLMs) like Gemini involves mechanisms like reinforcement learning from human feedback, which can lead to biases and quirky responses.
  2. Concerns arise about the use of LLMs for automated content moderation and the potential impact on historical and political education for children.
  3. The shift within Big Tech towards paternalistic content moderation reflects a move away from the libertarian culture predominant until the mid-2010s, highlighting evolving perspectives on regulating information online.

Import AI 362: Amazon's big speech model; fractal hyperparameters; and Google's open models

Import AI • 299 implied HN points • 26 Feb 24
🕹 Technology Language Models
  1. The full capabilities of today's AI systems are still not fully explored, with emerging abilities seen as models scale up.
  2. Google released Gemma, small but powerful AI models that are openly accessible, contributing to the competitive AI landscape.
  3. Understanding hyperparameter settings in neural networks is crucial as the fine boundary between stable and unstable training is found to be fractal, impacting the efficiency of training runs.

Import AI 359: $1 billion gov supercomputer; Apple’s good synthetic data technique; and a thousand-year old data library

Import AI • 339 implied HN points • 05 Feb 24
🕹 Technology Language Models
  1. Google uses LLM-powered bug fixing that is more efficient than human fixes, highlighting the impact of AI integration in speeding up processes.
  2. Yoshua Bengio suggests governments invest in supercomputers for AI development to stay ahead in monitoring tech giants, emphasizing the importance of AI investment in the public sector.
  3. Microsoft's Project Silica showcases a long-term storage solution using glass for archiving data, which is a unique and durable alternative to traditional methods.
  4. Apple's WRAP technique creates synthetic data effectively by rephrasing web articles, enhancing model performance and showcasing the value of incorporating synthetic data in training.

The Rise of Indian Llamas

Sector 6 | The Newsletter of AIM • 399 implied HN points • 25 Dec 23
🕹 Technology Language Models
  1. Llama 2 is a popular open-source language model with many downloads worldwide. In India, people are using it to create models that work well for local languages.
  2. A new Hindi language model called OpenHathi has been released, which is based on Llama 2. It offers good performance for Hindi, similar to well-known models like GPT-3.5.
  3. There is a growing interest in using these language models for business in India, indicating that the trend of 'Local Llamas' is just starting to take off.

Import AI 349: Distributed training breaks AI policy; turning GPT4 bad for $245; better weather forecasting through AI

Import AI • 459 implied HN points • 20 Nov 23
🕹 Technology Language Models
  1. Graph Neural Networks are used to create an advanced weather forecasting system called GraphCast, outperforming traditional weather simulation.
  2. Open Philanthropy offers grants to evaluate large language models like LLM agents for real-world tasks, exploring potential safety risks and impacts.
  3. Neural MMO 2.0 platform enables training AI agents in complex multiplayer games, showcasing the evolving landscape of AI research beyond language models.

Import AI 338: Consciousness and AI; self-improving language models; maps of thought.

Import AI • 539 implied HN points • 28 Aug 23
🕹 Technology Language Models
  1. Facebook introduces Code Llama, large language models specialized for coding, empowering more people with access to AI systems.
  2. DeepMind's Reinforced Self-Training (ReST) allows faster AI model improvement cycles by iteratively tuning models based on human preferences, but overfitting risks need careful management.
  3. Researchers identify key indicators from studies on human and animal consciousness to guide evaluation of AI's potential consciousness, stressing the importance of caution and a theory-heavy approach.

Import AI 342: Mistral dumps an LLM on BitTorrent; AMD vs NVIDIA; Sutton joins keen

Import AI • 539 implied HN points • 02 Oct 23
🕹 Technology Language Models
  1. AI startup Lamini is offering an 'LLM superstation' using AMD GPUs, challenging NVIDIA's dominance in AI chip market.
  2. AI researcher Rich Sutton has joined Keen Technologies, indicating a strong focus on developing Artificial General Intelligence (AGI).
  3. French startup Mistral released Mistral 7B, a high-quality open-source language model that outperforms other models, sparking discussions on safety measures in AI models.

The Automation Paradox: How More Tech Can Mean More Human Challenges

UX Psychology • 297 implied HN points • 12 Jan 24
🕹 Technology Language Models
  1. Increased automation can lead to unexpected complications for human tasks, creating a paradox where reliance on technology may actually hinder human performance.
  2. The 'Irony of Automation' highlights unintended consequences like automation not reducing human workload, requiring more complex skills for operators, and leading to decreased vigilance.
  3. Strategies like enhancing monitoring systems, maintaining manual and cognitive skills, and thoughtful interface design are crucial for addressing the challenges posed by automation and keeping human factors in focus.

Import AI 341: Neural nets can smell; technofeudalism via AI; China releases another solid open access model

Import AI • 459 implied HN points • 25 Sep 23
🕹 Technology Language Models
  1. China released open access language models trained on both English and Chinese data, emphasizing safety practices tailored to China's social context.
  2. Google and collaborators created a digital map of smells, pushing AI capabilities to not just recognize visual and audio data but also scents, opening new possibilities for exploration and understanding.
  3. An economist outlines possible societal impacts of AI advancement, predicting a future where superintelligence prompts dramatic changes in governance structures, requiring adaptability from liberal democracies.

Import AI 321: Open source GPT3; giving away democracy to AGI companies; GPT-4 is a political artifact

Import AI • 599 implied HN points • 20 Mar 23
🕹 Technology Language Models
  1. AI startup Assembly AI developed Conformer-1 using scaling laws for speech recognition domain, achieving better performance than other models.
  2. The announcement of GPT-4 by OpenAI signifies a shift towards a new political era in AI, raising concerns on the power wielded by private sector companies over AGI development.
  3. James Phillips highlights concerns over Western governments relinquishing control of AGI to US-owned private sector, proposing steps to safeguard democratic control over AI development.

Import AI 329: Compute IS data; don't build AI agents; AI needs a precautionary principle

Import AI • 399 implied HN points • 15 May 23
🕹 Technology Language Models
  1. Building AI scientists to advise humans is a safer alternative to building AI agents that act independently
  2. There is a need for a precautionary principle in AI development to address threats to democracy, peace, safety, and work
  3. Approaches like Self-Align show the potential for AI systems to self-bootstrap using synthetic data, leading to more capable models

LLMs and the "not" problem

The Counterfactual • 119 implied HN points • 19 Mar 24
🕹 Technology Language Models
  1. LLMs, like ChatGPT, struggle with negation. They often don't understand requests to remove something from an image and can still include it.
  2. Human understanding of negation is complex, as people process negative statements differently than positive ones. We might initially think about what is being negated before understanding the actual meaning.
  3. Giving LLMs more time to think, or breaking down their reasoning, can improve their performance. This shows that they might need support to mimic human understanding more closely.

Import AI 325: Automated mad science; AI vs democracy; and a 12B parameter language model

Import AI • 419 implied HN points • 17 Apr 23
🕹 Technology Language Models
  1. Prompt injection could be a major security risk in AI systems, making them vulnerable to unintended actions and compromising user privacy.
  2. The concentration of AI development in private companies poses a threat to democracy, as these language models encode the normative intentions of their creators without democratic oversight.
  3. The rapid race to build 'god-like AI' in the private sector is raising concerns about the lack of understanding and oversight, with experts warning about potential dangers to humanity.

The Latent State Window and the Limits of Reasoning in LLMs

jonstokes.com • 164 implied HN points • 05 Jul 25
🕹 Technology Language Models
  1. LLMs have limits when it comes to reasoning. If a problem is too complex or involves too many moving parts, the model can struggle to find a solution.
  2. The size of a language model's 'latent state window' matters. This window limits how much information the model can hold while trying to reason, separating it from just the number of tokens it can handle.
  3. To get good results from LLMs, it's best to keep tasks simple and broken down into manageable pieces. If you give the model too much to juggle at once, it won't perform well.

ChatGPT takes the Big Five Inventory

Vectors of Mind • 294 implied HN points • 27 Mar 23
🕹 Technology Language Models
  1. A language model like ChatGPT can take personality tests like the Big Five Inventory.
  2. ChatGPT's personality leans towards being conscientious and non-neurotic.
  3. It's fascinating how language models like ChatGPT can generate responses to personality test questions based on their programming and training.

Chain of Thought Prompting for LLMs

Deep (Learning) Focus • 294 implied HN points • 24 Apr 23
🕹 Technology Language Models
  1. CoT prompting leverages few-shot learning in LLMs to improve their reasoning capabilities, especially for complex tasks like arithmetic, commonsense, and symbolic reasoning.
  2. CoT prompting is most beneficial for larger LLMs (>100B parameters) and does not require fine-tuning or extensive additional data, making it an easy and practical technique.
  3. CoT prompting allows LLMs to generate coherent chains of thought when solving reasoning tasks, providing interpretability, applicability, and computational resource allocation benefits.

Import AI 322: Huawei's trillion parameter model; AI systems as moral patients; parasocial bots via Character.ai

Import AI • 399 implied HN points • 27 Mar 23
🕹 Technology Language Models
  1. Regulators advise against using AI to deceive people and emphasize the importance of mitigating any potential deception
  2. Huawei trains a trillion parameter model but may need more training on a larger dataset for optimal performance
  3. Researchers create a multimodal dialog model that incorporates visual cues to improve dialogue generation, suggesting advancements in AI's ability to understand and respond to context