The hottest Data Analysis Substack posts right now

And their main takeaways
Category
Top Technology Topics

Introducing the Model Memo

Artificial Ignorance • 25 implied HN points • 06 Mar 25
🕹 Technology AI Models Machine Learning Software Development Technology Trends Data Analysis
  1. Several new advanced AI models have been released recently, improving reasoning and knowledge. These models, like OpenAI's GPT-4.5 and Google's Gemini 2.0, excel in different areas.
  2. AI is becoming more interactive with features that let it browse the web and perform tasks for users. This shows a shift towards AI that can take action, not just chat.
  3. The best AI models now cost more, with some requiring premium subscriptions. While powerful models like GPT-4.5 have high access fees, other new features may be available for free with some limits.

HN blogs - 1/11/24

HackerNews blogs newsletter • 59 implied HN points • 02 Nov 24
🕹 Technology Software Engineering Data Analysis Cybersecurity Programming
  1. Measuring technical debt is crucial for leaders, especially CTOs. It helps in understanding and managing the challenges in software development.
  2. Freezing CEO salaries during layoffs can create a fairer work environment. It shows accountability and may protect jobs for regular employees.
  3. Life shouldn't solely be based on statistics. Everyone's experiences are unique and can't be fully represented by numbers.

MAMLM as a General Purpose Technology: The Ghost in the GDP Machine

Brad DeLong's Grasping Reality • 130 implied HN points • 24 Jun 25
🕹 Technology Artificial Intelligence Innovation Productivity Data Analysis Economic Impact
  1. Big technology changes, like AI, often take longer to have an impact than we expect. History shows that these changes usually happen in small steps instead of all at once.
  2. The way AI is being used in businesses is growing, with more companies starting to adopt these technologies. This can lead to higher productivity over time.
  3. To really benefit from new technologies like AI, we need patience and creativity in our systems. The changes won't happen overnight, but it's important to stick with it.

Scribble-based forecasting and AI 2027

DYNOMIGHT INTERNET NEWSLETTER • 562 implied HN points • 30 Jun 25
🕹 Technology AI Forecasting Data Analysis Machine Learning Simulation
  1. Both math and intuition can be used for forecasting, but they serve different purposes. Sometimes, using intuition can be more practical when creating predictions about complex situations.
  2. Math-based forecasts are best when the rules of a situation are well understood and complex. For simpler scenarios, basic predictions may be just as effective.
  3. Creating simple visual predictions, like drawing lines, can help clarify your thoughts. It's a great exercise to explore different potential outcomes and express predictions clearly.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

A use-theory of testing

arg min • 436 implied HN points • 24 Oct 24
🔬 Science Statistics Research Mathematics Data Analysis Theory
  1. Statistical tests are designed to help separate real signals from random noise. It's not just about understanding what they mean, but what they can do in practical situations.
  2. Many people misuse statistical tests, which can lead to misunderstandings about their purpose. Communities should establish clear guidelines on how to use these tests correctly.
  3. The main function of statistical tests is to regulate opinions and decisions in various fields like tech and medicine. They help ensure that important standards are met, rather than just preventing errors.

What Is Statistics' Purpose?

arg min • 734 implied HN points • 14 Oct 24
🔬 Science Statistics Research Medicine Philosophy Data Analysis
  1. Statistics should help us test claims by measuring how surprising the results are. However, there's doubt about whether our current statistical tests actually do this well.
  2. Randomized trials are important because they help us learn about treatments that may not always work. They focus on safety as much as they do on finding effective solutions.
  3. The field of statistics needs to be clear about its purpose. We should distinguish between using statistics for proving theories and for practical decision-making like quality control.

The Shape of Stats to Come

arg min • 634 implied HN points • 10 Oct 24
🚌 Education Statistics Optimization Philosophy Data Analysis Mathematics
  1. Statistics often involves optimizing methods to get the best results. Many statistical techniques can actually be viewed as optimization problems.
  2. Choosing a statistical method isn't just about the math—it's also based on beliefs about reality. This philosophical side is important but often overlooked.
  3. There's a danger in relying too much on tools and models we can solve. Sometimes, we force the data to fit our preferred methods instead of being open to the actual complexities.

ChatGPT in Shambles

Marcus on AI • 13161 implied HN points • 04 Feb 25
🕹 Technology Artificial Intelligence Machine Learning Natural Language Processing Data Analysis Software Development
  1. ChatGPT still has major reliability issues, often providing incomplete or incorrect information, like missing U.S. states in tables.
  2. Despite being advanced, AI can still make basic mistakes, such as counting vowels incorrectly or misunderstanding simple tasks.
  3. Many claims about rapid progress in AI may be overstated, as even simple functions like creating tables can lead to errors.

AI analysis: ChatGPT vs Claude

Handy AI • 19 implied HN points • 29 Oct 24
🕹 Technology AI Data Analysis Machine Learning Software Development Information Technology
  1. ChatGPT performed better in analyzing a Spotify dataset, providing accurate insights without errors, and displaying clear visualizations.
  2. Claude encountered issues with text extraction and made mistakes in data interpretation, like incorrectly assigning genre labels where they didn't exist in the dataset.
  3. Overall, ChatGPT offered a smoother user experience, allowing users to follow along with the analysis while Claude's process was less straightforward.

Your Lab Tests

Ground Truths • 15921 implied HN points • 14 Dec 24
🏥 Health & Wellness Lab Tests Data Analysis
  1. Your individual lab results, like the Complete Blood Count (CBC), can vary a lot between people but stay stable for you over time. This means your personal health data can give more accurate insights than just average values used for everyone.
  2. Personalized reference values from CBC tests can help predict health risks better than conventional methods. They show clearer connections to potential diseases and can indicate specific health issues.
  3. Using advanced technology like AI to analyze these personal health metrics could help doctors spot risks early. This approach can enhance patient care by identifying high-risk individuals for proactive health management.

Checking in on the 2025 Cyber 150

The Security Industry • 10 implied HN points • 03 Jul 25
🕹 Technology Cybersecurity Startups Investment Data Analysis Market Trends
  1. The Cyber 150 list includes cybersecurity companies with between 50 to 500 employees, showcasing those on the rise before they grow too big.
  2. Funding is flowing into these companies, with some receiving over $100 million, totaling around $2.3 billion in the first half of 2025 alone.
  3. Companies that grow past 500 employees or fail to grow can graduate from or drop off the Cyber 150 list, highlighting their changing status in the industry.

DeepSeek-V3: Technical Details

Gonzo ML • 252 implied HN points • 06 Feb 25
🕹 Technology Artificial Intelligence Machine Learning Computer Science Data Analysis Software Development
  1. DeepSeek-V3 uses a new technique called Multi-head Latent Attention, which helps to save memory and speed up processing by compressing data more efficiently. This means it can handle larger datasets faster.
  2. The model incorporates an innovative approach called Multi-Token Prediction, allowing it to predict multiple tokens at once. This can improve its understanding of context and boost overall performance.
  3. DeepSeek-V3 is trained using advanced hardware and new training techniques, including utilizing FP8 precision. This helps in reducing costs and increasing efficiency while still maintaining model quality.

Freddie Mac House Price Index Declined in May; Up 2.2% Year-over-year

CalculatedRisk Newsletter • 28 implied HN points • 30 Jun 25
💰 Finance Real Estate Market Trends Investment Economic Indicators Data Analysis
  1. The Freddie Mac House Price Index dropped by 0.23% in May but is still up 2.2% compared to last year. This shows that while prices are currently declining, there has been some growth over the past year.
  2. Florida and Texas are experiencing significant price declines, with 17 out of the 30 cities with the biggest drops located in these states. This trend indicates that real estate markets in these areas are facing challenges.
  3. Overall, 31 states and Washington D.C. have seen house prices fall since their peak. With inventory increasing and low sales, housing price growth may slow down even more in the future.

SQL and Python Combined Change RevOps

beyondrevenueoperations • 19 implied HN points • 27 Oct 24
💼 Business Data Analysis Automation Technology Integration Strategic Planning
  1. Combining SQL and Python makes data management much easier. SQL helps you access and pull data, while Python helps analyze it and create reports.
  2. Using SQL, you can break down data silos from different systems to get a complete view of your customers and performance. This is crucial for making smart, data-driven decisions.
  3. With Python, you can automate tasks, build predictive models, and visualize data, which saves time and enhances your ability to understand trends and insights.

OSD 330: The rise of sensorship

Open Source Defense • 28 implied HN points • 17 Jun 25
🕹 Technology Sensors Innovation Gadgets Measurement Data Analysis
  1. Sensors help us understand and measure things better. The more accurate our sensors are, the more we can improve our products and practices.
  2. In different fields, the use of sensors is at various stages. Some areas, like competition shooting, are advanced, while others, like non-lethal weapons, have much room for growth.
  3. Using objective measurements can change our understanding of different situations. By having clear data, we can make better decisions and improve our overall knowledge.

We must fearlessly pursue truth, and truth alone

Heterodox STEM • 192 implied HN points • 08 Jun 25
🔬 Science Climate change Research Integrity Data Analysis Environmental Science
  1. It's important for scientists to report accurate information about climate change, without exaggerating risks. This helps keep trust in science strong.
  2. Some published studies can mislead people by only showing negative aspects of climate data. A balanced view is necessary to understand the full picture.
  3. We should encourage honest and thorough research that combines both good and bad news about our world. Pursuing the truth is essential.

What Homes Are Missing, again?

Erdmann Housing Tracker • 231 implied HN points • 03 Feb 25
💼 Business Housing Economics Data Analysis Market Trends Population Growth
  1. There is a significant shortage of homes in the U.S., estimated at around 15 million. This is due to various factors like vacancies and the rising number of adults per home.
  2. Vacancies have dropped over the years, and we might be short about 5 million vacant units needed to keep rent inflation stable.
  3. Population growth has slowed since 2008 and has likely affected housing demand, which adds pressure to the existing housing shortage.

“Nvidia could soon take a serious hit, too”

Marcus on AI • 4228 implied HN points • 27 Jan 25
🕹 Technology AI Market Trends Economics Tech Companies Data Analysis
  1. Nvidia's stock might be facing a big drop, which is a concern for investors. A decline over 10% indicates that something is going on in the market.
  2. The market can behave in unpredictable ways, and this uncertainty can be tough for investors to manage. Today might be a key moment in the stock market.
  3. Overall, the economics of generative AI can lead to unexpected changes, making it a wild area to watch for investors and tech enthusiasts.

When will psychiatry invent the barcode?

Wood From Eden • 1344 implied HN points • 04 Dec 24
🏥 Health Politics Mental health Psychiatry Diagnosis Research Data Analysis
  1. Psychiatry has a problem with labels. Many old labels have been removed without clear replacements, making research and understanding harder.
  2. Using numbers instead of words could help describe a person's mental health better. A barcode-like system could show traits and abilities at a glance.
  3. Psychology is subjective and changes over time. Collecting more data through tests can help improve understanding and research in mental health.

The Most Major Hurricanes Ever

The Honest Broker Newsletter • 2973 implied HN points • 27 Jan 25
🌞 Climate & Environment Climate change Weather Patterns Natural Disasters Data Analysis
  1. In 2024, there were a lot of major hurricanes, tying with 2015 for the highest since records began, which raises questions about climate patterns.
  2. Despite the increase in hurricane landfalls, there hasn't been a clear trend showing that hurricanes are becoming more intense or frequent over time.
  3. Experts believe that while human activity may influence hurricanes, detecting these changes amidst natural variability is very challenging.

COVID time series graphs show clearly the COVID vaccine kill people. That's why they keep the plots hidden from view.

Steve Kirsch's newsletter • 9 implied HN points • 11 Jun 25
🏥 Health Politics Public Health Vaccines Safety Data Analysis Medical Ethics
  1. Time series graphs can show if a vaccine is safe or not by plotting daily deaths after vaccination. A safe vaccine should show a flat line after the initial period.
  2. Current data for COVID vaccines shows increasing mortality rates after vaccination, which suggests they may not be safe. Many reports don’t show this data.
  3. The medical community often ignores clear signs of vaccine risks, despite evidence appearing in graphs and reports, leading to frustration among those who analyze the data.

The new AI scaling law shell game

Marcus on AI • 4663 implied HN points • 24 Nov 24
🕹 Technology Artificial Intelligence Machine Learning Computing Data Analysis
  1. Scaling laws in AI aren't as reliable as people once thought. They're more like general ideas that can change, rather than hard rules.
  2. The new approach to scaling, which focuses on how long you train a model, can be costly and doesn't always work better for all problems.
  3. Instead of just trying to make existing models bigger or longer-lasting, the field needs fresh ideas and innovations to improve AI.

A Pfizer shot. A failed heart. A transplant. Get vaxxed again?

RESCUE with Michael Capuzzo • 9787 implied HN points • 08 Jun 23
🏥 Health Politics Vaccination Data Analysis
  1. John Berndsen's heart complications after receiving the Pfizer vaccine illustrate a potential link to myocarditis and the importance of questioning vaccine safety.
  2. Many adverse reactions to COVID-19 vaccines are not being reported in the media, and the numbers show a significant impact on health, including deaths.
  3. John Berndsen's experience highlights the importance of critically examining the safety and necessity of additional vaccine doses, especially for vulnerable individuals.

TSMC v Intel in Two Charts

Tim Culpan’s Position • 119 implied HN points • 05 Sep 24
🕹 Technology Semiconductors Charts Data Analysis Market Trends
  1. TSMC and Intel are two major players in the semiconductor industry. Their performance and strategies have crucial implications for technology.
  2. Visual data can highlight important differences in the technical and financial health of these companies. Charts can make complex information easier to understand.
  3. Recent reports show that Intel is facing significant challenges, while TSMC continues to lead in production and technology advancements. This could shape the future of the tech industry.

Deconstructing the Transformers ReAct JSON System Prompt

Encyclopedia Autonomica • 39 implied HN points • 13 Oct 24
🕹 Technology AI Software Machine Learning Data Analysis Development
  1. Transformers use a specific structure for commands called JSON. This makes it easier to describe actions clearly and effectively.
  2. The system prompt includes rules that the agent must follow, like focusing on one action at a time and using the correct values for inputs.
  3. The design also emphasizes iterative reasoning, where the agent can build on previous observations to make better decisions in tasks.

Conservatives Are Lying on Immigrant Crime

Richard Hanania's Newsletter • 3657 implied HN points • 07 Oct 24
🇺🇸 U.S. Politics Immigration Policy Crime Rates Public Discourse Political Rhetoric Data Analysis
  1. Many people incorrectly believe that immigration leads to higher crime rates. In reality, data shows that most immigrants, especially legal ones, tend to commit less crime than native-born citizens.
  2. Some politicians use scary language about immigrants increasing crime to push their agenda. This can create a false narrative that makes the public fearful and misinformed about the actual impact of immigration.
  3. Immigrants often face more crime themselves and can actually help reduce crime rates in communities by starting businesses and contributing to the economy. So, they can serve as a buffer against crime rather than a cause of it.

Measuring Programmer Influence, Kinda Sorta

Software Design: Tidy First? • 1347 implied HN points • 27 Jan 25
🕹 Technology Software Development Data Analysis Programming Project management
  1. Data can provide hints about a programmer's influence, but it can't give a clear answer. It's important to interpret the data with caution and avoid making strict decisions based solely on it.
  2. Creating files is one way to measure initiation of influence, but it's not the only factor. The impact is also determined by how frequently those files are modified by others.
  3. Using data for bonuses or promotions can lead to problems. It's better to focus on improvement and impact rather than just the numbers, to maintain a healthy team dynamic.

Who Predicted 2023?

Astral Codex Ten • 8534 implied HN points • 05 Mar 24
🕹 Technology Forecasting Prediction Contest Algorithm Data Analysis
  1. The Annual Forecasting Contest on astralcodexten.com involves participants making predictions about various questions, helping to determine if one identifiable genius or aggregated mathematical predictions work best for foreseeing the future.
  2. The winners of the contest were both amateurs and seasoned forecasting veterans, showcasing a mix of skill and luck in predicting outcomes.
  3. Metaculus outperformed prediction markets, superforecasters, and the wisdom of crowds in the contest, suggesting that consistent high performance might be rare but achievable with specific methods like those used by superforecaster Ezra Karger.

STFU

Klement on Investing • 4 implied HN points • 20 Jun 25
😂 Humor Social Commentary Anecdotes Data Analysis Gender differences
  1. On average, women speak more words per day than men. Women use about 13,349 words while men use around 11,950 words daily.
  2. As people age, how much they talk can change. Younger men and women talk similarly, but older men often become more talkative than older women.
  3. Some people barely talk, while others can speak a ton, like 50,000 words a day. It's interesting to see such a big range in how much different people communicate.

🤘ACDC (not that one)

Gonzo ML • 63 implied HN points • 29 Jan 25
🕹 Technology Artificial Intelligence Machine Learning Neural Networks Data Analysis Automation
  1. The paper introduces a method called ACDC that automates the process of finding important circuits in neural networks. This can help us better understand how these networks work.
  2. Researchers follow a three-step workflow to study model behavior, and ACDC fully automates the last step which helps identify connections that matter for a specific task.
  3. While ACDC shows promise, it isn't perfect. It may miss some important connections and needs adjustments for different tasks to improve its accuracy.

LinkedIn

Public Universal Friend • 79 implied HN points • 02 Sep 24
💼 Business Marketing Customer Engagement Data Analysis Growth Strategies
  1. Using a customer engagement platform like Customer.io can help marketers improve their targeting and maximize growth. It offers better data management and less need for technical support.
  2. Spring is a great time for businesses to focus on improving conversions through digital marketing strategies. Real-time data can help companies get more return on their investment.
  3. Personal connections and genuine interactions are valuable, even in business communication. Taking the time to show real interest can make a difference.

Independent SAGE has joined substack!

Independent SAGE continues • 1418 implied HN points • 20 Mar 24
🏥 Health Politics Public Health Pandemic response Vaccination Data Analysis
  1. Independent SAGE has launched a Substack to share insights about Covid research and data. They aim to provide valuable information directly from experts to the public.
  2. They plan to post updates roughly every two weeks, including responses to important new research and news. This helps keep everyone informed about the ongoing situation.
  3. The Substack will remain free for subscribers, encouraging more people to stay updated on Covid developments and public health measures.

AI Hallucinations on the Decline

Jakob Nielsen on UX • 21 implied HN points • 13 Feb 25
🕹 Technology AI Usability UX Design Data Analysis
  1. AI models are getting better at reducing false information, called hallucinations. This means they are less likely to make things up over time.
  2. Bigger AI models generally make fewer mistakes. As AI technology improves, we can expect even fewer errors from future models.
  3. While waiting for better AI, improving user experience can help users spot and double-check misleading information, making it easier to trust AI outputs.

Oxford Nanopore Customer Growth?

ASeq Newsletter • 14 implied HN points • 20 Jun 25
🕹 Technology Biotech Research Data Analysis Industry Trends Innovation
  1. Oxford Nanopore has stopped sharing details about its customer base, which raises concerns about growth. It's unclear how many customers they really have now.
  2. The MinION, which has a lot of users, isn't very profitable for Oxford, so its slowing growth might not be a big issue.
  3. Research funding seems to be declining overall, which could affect Oxford and other companies in the field, especially for their larger customers.

Background Work

Software Design: Tidy First? • 1568 implied HN points • 28 Oct 24
🕹 Technology Software Design Programming Engineering Data Analysis Artificial Intelligence
  1. Background work is doing extra research or tasks beyond what's necessary. It's a way to learn and grow your skills.
  2. Successful programmers often engage in background work, which helps them become more knowledgeable and credible.
  3. While background work can sometimes feel like extra effort, it usually pays off quickly and can save time in the long run.