The hottest Data science Substack posts right now

And their main takeaways
Category
Top Technology Topics

Phi-3, your Pocket LLM!

Aziz et al. Paper Summaries • 79 implied HN points • 29 Apr 24
🕹 Technology Data science
  1. Microsoft's Phi-3 is a new AI model that is small enough to run on your phone, yet still performs well. This is a big deal because most AI models are too large for personal devices.
  2. The model uses high-quality, filtered data for training, focusing on reasoning and educational materials. This approach makes Phi-3 better at understanding rather than just memorizing facts.
  3. Even though Phi-3 is powerful, it has some limitations, like not being multilingual. There are also tasks it struggles with, like those needing lots of factual knowledge.

The Sequence Radar #696: Google AI Ultra’s New Gold-Medal Reasoning Model is Available

TheSequence • 119 implied HN points • 03 Aug 25
🕹 Technology Data science
  1. Google released a new AI model called Gemini 2.5 Deep Think that can solve complex math problems like a human. It performed so well that it won a gold medal at the International Math Olympiad.
  2. This model uses advanced strategies to explore many possible solutions at once, making it faster and more creative than previous AIs.
  3. The emergence of such powerful AI means we need to discuss how to use these systems responsibly, ensuring they benefit everyone and maintain fair access.

The Sequence Research #673: Infinite Self-Improvement: Unpacking Sakana's Darwin Gödel Machine

TheSequence • 154 implied HN points • 27 Jun 25
🕹 Technology Data science
  1. The Darwin Gödel Machine (DGM) is a new kind of AI that can change its own code to improve. It combines two ideas: self-modifying machines and evolving through trial and error.
  2. Instead of needing complicated proofs for changes, DGM tests its code edits under real-world conditions. This helps it learn quickly and safely from what works.
  3. DGM has shown significant improvement in coding benchmarks, outperforming humans and traditional methods. This means it can continually get better at coding and solving problems.

LLM Links, 1/27/2025

In My Tribe • 318 implied HN points • 27 Jan 25
🕹 Technology Data science
  1. AI is improving quickly, making it easier for students to answer essay questions by providing high-quality responses from various texts. This change may reduce the value of traditional essay exams.
  2. A World Bank project in Nigeria successfully used AI in education, enhancing learning equivalent to nearly two years in just six weeks. This shows promise for AI to help education in underdeveloped areas.
  3. OpenAI is developing AI models to transform science, including engineering proteins that enhance cellular functions. This could lead to significant advancements in fields like bioengineering.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Holy Grails of Data: Self-Service, Single Truths, and the Role of AI

SeattleDataGuy’s Newsletter • 365 implied HN points • 27 Dec 24
🕹 Technology Data science
  1. Self-service analytics is still a goal for many companies, but it often falls short. Users might struggle with the tools or want different formats for the data, leading to more questions instead of fewer.
  2. Becoming truly data-driven is a challenge for many organizations. Trust issues with data, preference for gut feelings, and poor communication often get in the way of making informed decisions.
  3. People need to be data literate for businesses to succeed with data. The data team must present insights clearly, while business teams should understand and trust the data they work with.

The Importance Of Granular Data Design For Fine-Tuning

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 59 implied HN points • 02 May 24
🕹 Technology Data science
  1. Granular data design helps improve the behavior and abilities of language models. This means making training data more specific so the models can reason better.
  2. New methods like Partial Answer Masking allow models to learn self-correction. This helps them improve their responses without needing perfect answers in the training data.
  3. Training models with a focus on long context helps them retrieve information more effectively. This approach tackles issues where models can lose important information in lengthy input.

Data Science Weekly - Issue 510

Data Science Weekly Newsletter • 279 implied HN points • 31 Aug 23
🕹 Technology Data science
  1. Autonomous drones can now race at human champion levels using deep reinforcement learning. This shows how advanced technology can mimic skilled human behavior in competitive sports.
  2. Google is rapidly developing its AI capabilities and plans to surpass GPT-4 by a significant margin soon. This could lead to more powerful AI tools for various applications.
  3. Reinforced Self-Training (ReST) is a new method for improving language models by aligning their outputs with human preferences. It offers better translation quality and can be done efficiently with less data.

Diffusion Models are Evolutionary Algorithms

Gonzo ML • 441 implied HN points • 09 Nov 24
🕹 Technology Data science
  1. Diffusion models and evolutionary algorithms both involve changing data over time through processes like selection and mutation, which can lead to new and improved results.
  2. The new algorithm called Diffusion Evolution can find multiple good solutions at once, unlike traditional methods that often focus on one single best solution.
  3. There are exciting connections between learning and evolution, hinting that they may fundamentally operate in similar ways, which opens up many questions about future AI developments.

Tülu 3: The next era in open post-training

Democratizing Automation • 404 implied HN points • 21 Nov 24
🕹 Technology Data science
  1. Tulu 3 introduces an open-source approach to post-training models, allowing anyone to improve large language models like Llama 3.1 and reach performance similar to advanced models like GPT-4.
  2. Recent advances in preference tuning and reinforcement learning help achieve better results with well-structured techniques and new synthetic datasets, making open post-training more effective.
  3. The development of these models is pushing the boundaries of what can be done in language model training, indicating a shift in focus towards more innovative training methods.

Moneyballocracy

Gradient Ascendant • 20 implied HN points • 22 Dec 25
🕹 Technology Data science
  1. AI models are rapidly getting good at forecasting and already rival the wisdom of crowds and some human forecasters.
  2. Forecasting with AI is cheap and scalable, so you can run detailed, conditional predictions across thousands of stocks, counties, or scenarios that used to be impractical.
  3. Making the future more legible will reshape elections and politics: it can help match policy to voter preferences but also enable targeted manipulation, and any side that uses it effectively will gain a real advantage.

The Sequence Knwoledge #776: Fake It 'Til You Make It: How RL is Perfecting Synthetic Data.

TheSequence • 21 implied HN points • 23 Dec 25
🕹 Technology Data science
  1. Reinforcement learning environments can manufacture synthetic data by letting agents interact with simulators or APIs, producing richly labeled trajectories of states, actions, rewards, failures, and recoveries.
  2. This method is especially valuable when real data is scarce or privacy-restricted, and it shines in domains with verifiable outcomes like coding sandboxes, web automation, spreadsheets/SQL, and robotics-in-sim.
  3. Executing tasks to generate data (instead of just describing answers) gives models supervision on how to act and recover, and techniques like Reflexion can use those RL-generated trajectories to iteratively improve agents.

Speculative RAG By Google Research

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 19 implied HN points • 12 Jul 24
🕹 Technology Data science
  1. Retrieval Augmented Generation (RAG) is a way to improve answers by using a mix of information from language models and external sources. By doing this, it gives more accurate and timely responses.
  2. The new Speculative RAG method uses a smaller model to quickly create drafts from different pieces of information, letting a larger model check those drafts. This makes the whole process faster and more effective.
  3. Using smaller, specialized language models for drafting helps save on costs and reduces wait times. It can also improve the accuracy of answers without needing extensive training.

Embracing the New Era of Accelerated Testing - Issue 150

Data Analysis Journal • 235 implied HN points • 28 Jun 23
🕹 Technology Data science
  1. Embracing accelerated testing in the modern data analysis landscape is essential for success.
  2. The current traditional academic workflow for A/B testing may not be suitable for the evolving landscape of experimentation.
  3. To thrive in the era of rapid feature flagging and A/B testing, teams need to adapt by automating statistical checks, simplifying documentation, and eliminating bias.

Measuring the "readability" of texts with Large Language Models

The Counterfactual • 119 implied HN points • 02 Feb 24
🕹 Technology Data science
  1. Readability is how easy it is to understand a text. It matters in many areas like education, manuals, and legal documents.
  2. Traditional readability formulas like Flesch-Kincaid are simple but not enough. New methods that consider more linguistic features are being developed for better accuracy.
  3. Using large language models like GPT-4 can give good estimates of text readability. In one study, GPT-4's scores were better than traditional methods in predicting human readability judgments.

Did OpenAI Just Solve Abstract Reasoning?

AI: A Guide for Thinking Humans • 344 implied HN points • 23 Dec 24
🕹 Technology Data science
  1. OpenAI's new model, o3, showed impressive results on tough reasoning tasks, achieving accuracy levels that could compete with human performance. This signals significant advancements in AI's ability to reason and adapt.
  2. The ARC benchmark tests how well machines can recognize and apply abstract rules, but recent results suggest some solutions may rely more on extensive compute than true understanding. This raises questions about whether AI is genuinely learning abstract reasoning.
  3. As AI continues to improve, the ARC benchmark may need updates to push its limits further. New features could include more complex tasks and better ways to measure how well AI can generalize its learning to new situations.

Data Science Weekly - Issue 507

Data Science Weekly Newsletter • 279 implied HN points • 11 Aug 23
🕹 Technology Data science
  1. Large Language Models (LLMs) can take over some data tasks, but they won't replace all data jobs. Many tasks still need human insight and specialized skills.
  2. Understanding machine learning theory takes a long time, but in the industry, practical implementation is often more important. It's crucial to balance theory and hands-on skills.
  3. The new field of mechanistic interpretability is growing. Researchers are looking at how models learn and generalize, aiming to make sense of how AI works.

JAX things to watch for in 2025

Gonzo ML • 378 implied HN points • 26 Nov 24
🕹 Technology Data science
  1. The new NNX API is set to replace the older Linen API for building neural networks with JAX. It simplifies the coding process and offers better performance options.
  2. The shard_map feature improves multi-device computation by allowing better handling of data. It’s a helpful evolution for developers looking for precise control over their parallel computing tasks.
  3. Pallas is a new JAX tool that lets users write custom kernels for GPUs and TPUs. This allows for more specialized and efficient computation, particularly for advanced tasks like training large models.

Data Science Weekly - Issue 502

Data Science Weekly Newsletter • 319 implied HN points • 07 Jul 23
🕹 Technology Data science
  1. Generative design is making strides in drug discovery, but there are still challenges to address for better outcomes.
  2. The UK government is investing in a Foundation Model Taskforce to harness AI for societal benefits and safety.
  3. Keeping updated with developments in data science, such as new models and applications, is essential for professionals in the field.

The Easier You Train The Faster You Finish

The Healthtech Initiative • 5 implied HN points • 09 Feb 26
🎾 Sports Data science
  1. More total mileage in the six-month build-up leads to faster marathon times — about 4.4 minutes quicker per extra 100 km.
  2. Spending lots of time at or above race pace can backfire — extra race-pace minutes were linked to slower finish times.
  3. A large amount of easy, low-intensity running pays off — roughly 1,000 more minutes of easy training was associated with about 7 minutes faster.

Data Science Weekly - Issue 535

Data Science Weekly Newsletter • 99 implied HN points • 23 Feb 24
🕹 Technology Data science
  1. Scaling AI tools like ChatGPT involves overcoming many engineering challenges to handle large user demands. It's important to manage growth effectively to keep users satisfied.
  2. There's a lot of information out there about generative AI, making it hard to keep up. A guidebook can help condense this information and provide practical insights.
  3. Linear regression is still a valuable tool in data science. Sometimes going back to basics can yield better results than relying on complex models.

TheSequence Radar #674: Transformers in the Genome: How AlphaGenome Reimagines AI-Driven Genomics

TheSequence • 133 implied HN points • 29 Jun 25
🕹 Technology Data science
  1. AlphaGenome is a new AI model that helps understand the genome better. It predicts various functions in DNA, enabling quick analysis of genetic variants.
  2. This model combines different types of data into one system, making it easier and faster to see how genetic changes might affect health.
  3. DeepMind is offering early access to AlphaGenome for researchers, encouraging collaboration between academia and industry to unlock more discoveries in genetics.

E-Mail Course On Conformal Prediction

Mindful Modeler • 479 implied HN points • 13 Dec 22
🚌 Education Data science
  1. Conformal prediction turns point predictions into prediction sets with a probability guarantee of covering the true outcome, working for any model without requiring a distribution assumption.
  2. The 5-week email course on conformal prediction offers a free, convenient way to learn about this uncertainty quantification method.
  3. Resources like Valeriy's list on conformal prediction and an academic introduction paper can be helpful for diving into and understanding conformal prediction.

Data Science Weekly - Issue 491

Data Science Weekly Newsletter • 419 implied HN points • 21 Apr 23
🕹 Technology Data science
  1. AI academics are facing challenges keeping up with private sector investments. It's important for them to find survival strategies to remain competitive.
  2. There are ongoing discussions about the rapid progress in machine learning and how it can be overwhelming for developers. Many are sharing thoughts on adapting to this fast-paced change.
  3. Visualizing neural networks properly can help clarify concepts. There is a push for better diagrams to avoid confusion in understanding how these networks function.

RAG, Hallucination & Structure: Research By ServiceNow

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 59 implied HN points • 18 Apr 24
🕹 Technology Data science
  1. ServiceNow is using a method called Retrieval-Augmented Generation (RAG) to help transform user requests in natural language into structured workflows. This aims to improve how easily users can create workflows without needing deep technical knowledge.
  2. By using RAG, they want to reduce 'hallucination', which is when AI generates wrong or irrelevant info, and make the AI more reliable. This is important for gaining user trust in AI systems.
  3. The study also suggests future improvements, like changing output formats for efficiency and streamlining processes so that users can see steps one at a time, making it easier to follow along.

Evaluating The Quality Of RAG & Long-Context LLM Output

Cobus Greyling on LLMs, NLU, NLP, chatbots & voicebots • 19 implied HN points • 08 Jul 24
🕹 Technology Data science
  1. Evaluating the performance of RAG and long-context LLMs is tough because there isn't a common task to compare them on. This makes it hard to know which system works better.
  2. Salesforce created a new way to test these models called SummHay, where they summarize information from large text collections. The results show that even the best models struggle to match human performance.
  3. RAG systems generally do better at citing sources, while long-context LLMs might capture insights more thoroughly but have citation issues. Choosing between them involves trade-offs.

AI and the future of weather forecasting

The PhilaVerse • 123 implied HN points • 02 Jul 25
🕹 Technology Data science
  1. AI is changing how we predict the weather by offering quicker and more efficient methods compared to traditional forecasting. This helps provide better updates, especially for things like storms and heatwaves.
  2. While AI forecasting models are fast, they currently work at a lower resolution than traditional systems. They still depend on traditional methods for some accurate initial data.
  3. There is growing interest worldwide in using AI for weather forecasting. This technology could improve disaster preparedness, agriculture, and energy management, making it valuable for many industries.

Accio Insights: The Marauder’s Map of the ML World

The Palindrome • 3 implied HN points • 19 Feb 26
🕹 Technology Data science
  1. Embeddings are learned, dense numerical vectors that capture what words or items mean in context instead of using one‑hot or random encodings.
  2. Similarity in embedding space is measured by the cosine of the angle between vectors, and relationships show up as directions you can add or subtract (for example, king − man + woman ≈ queen), so similar things cluster and outliers stand out.
  3. Embeddings are a core building block across ML systems — powering search, LLMs, image generators, and recommendations — and engineers must design around retrieval, scale, latency, and reliability when using them in production.

What are embeddings?

Normcore Tech • 1353 implied HN points • 07 Jun 23
🕹 Technology Data science
  1. The author delved deep into the concept of embeddings in deep learning.
  2. The author's journey in understanding embeddings involved a significant amount of research and work.
  3. The author hopes that others can benefit from their learning about embeddings as well.

Results from poll #4

The Counterfactual • 39 implied HN points • 21 May 24
🕹 Technology Data science
  1. The recent poll found that two topics, an explainer on interpretability and a guide to becoming an LLM-ologist, were equally popular among voters.
  2. The plan is to write about both topics in the coming months, keeping the content varied as usual.
  3. Two new papers were published this month, one on multimodal LLMs and another on Korean language models, highlighting ongoing research in these areas.

LLMs Fight With Both Hands Tied Behind Their Back

Am I Stronger Yet? • 313 implied HN points • 27 Dec 24
🕹 Technology Data science
  1. Large Language Models (LLMs) like o3 are becoming better at solving complex math and coding problems, showing impressive performance compared to human competitors. They can tackle hard tasks with many attempts, which is different from how humans might solve them.
  2. Despite their advances, LLMs struggle with tasks that require visual reasoning or creativity. They often fail to understand spatial relationships in images because they process information in a linear way, making it hard to work with visual puzzles.
  3. LLMs rely heavily on knowledge in their 'heads' and do not have access to real-world knowledge. When they gain access to more external tools, their performance could improve significantly, potentially changing how they solve various problems.

Singh & Sins of AI 💦

Sector 6 | The Newsletter of AIM • 99 implied HN points • 13 Feb 24
🕹 Technology Data science
  1. The Indian AI scene is growing, with many new language models being developed based on Meta's Llama 2. This shows a collaborative spirit in the open-source community.
  2. There are specific models being made for different Indian languages like Kannada, Telugu, Odia, and Tamil. These models help in making AI more accessible to people speaking these languages.
  3. There is a strong need for India to create its own unique open-source AI model. This would allow other developers to build on it rather than relying on external sources.

The Sequence Opinion #754: Generalist vs. Specialist: Which School Will Win in Mathematical AI

TheSequence • 35 implied HN points • 13 Nov 25
🕹 Technology Data science
  1. Generalist AI models can handle a wide range of math problems and can even score well on exams, but they struggle with creating new math concepts.
  2. Specialist AI models focus on specific math tasks and provide precise answers, but they have limits in flexibility and scope.
  3. Choosing between generalist and specialist models depends on the math task at hand, as each has its own strengths and weaknesses.

BI is not ready for AI

HyperArc • 3 HN points • 06 Sep 24
🕹 Technology Data science
  1. Business Intelligence (BI) needs both good models and great data to be effective with AI. Without quality data, AI can't really show its true power.
  2. Many BI tools only focus on successful outcomes, like specific metrics, while ignoring the complete journey of discovery. This limited data can lead to missing important insights.
  3. To improve AI's effectiveness in BI, we should include a wider range of experiences and exploration paths, not just successful queries. This fuller picture can help create better AI training sets.

Data Science Weekly - Issue 492

Data Science Weekly Newsletter • 379 implied HN points • 28 Apr 23
🕹 Technology Data science
  1. There is a new Slack community for paid subscribers focused on learning new tools and techniques in data science and career growth. It's a good place for support and sharing information.
  2. A/B testing is important for experiments and there are recommended resources to help design and run successful tests. Proper planning and communication are key to making A/B testing effective.
  3. Large Language Models (LLMs) are becoming more useful, and several resources are available for learning how to work with them. Understanding how they operate can help create valuable applications.

OpenAI Announces o1 Model And ChatGPT Pro ($200/Mo)

The Algorithmic Bridge • 329 implied HN points • 05 Dec 24
🕹 Technology Data science
  1. OpenAI has launched a new AI model called o1, which is designed to think and reason better than previous models. It can now solve questions more accurately and is faster at responding to simpler problems.
  2. ChatGPT Pro is a new subscription tier that costs $200 a month. It provides unlimited access to advanced models and special features, although it might not be worth it for average users.
  3. o1 is not just focused on math and coding; it's also designed for everyday tasks like writing. OpenAI claims it's safer and more compliant with their policies than earlier models.

Complete Summary of Absolute, Relative and Rotary Position Embeddings!

Aziz et al. Paper Summaries • 79 implied HN points • 31 Mar 24
🕹 Technology Data science
  1. Transformers can't understand the order of words, so position embeddings are used to give them that context.
  2. Absolute embeddings assign unique values to each word's position, but they struggle with new positions beyond what they trained on.
  3. Relative embeddings focus on the distance between words, which makes the model aware of their relationships, but they can slow down training and searching.