The hottest Data Analysis Substack posts right now

And their main takeaways
Category
Top Technology Topics

Introducing the Model Memo

Artificial Ignorance • 25 implied HN points • 06 Mar 25
🕹 Technology AI Models Machine Learning Software Development Technology Trends Data Analysis
  1. Several new advanced AI models have been released recently, improving reasoning and knowledge. These models, like OpenAI's GPT-4.5 and Google's Gemini 2.0, excel in different areas.
  2. AI is becoming more interactive with features that let it browse the web and perform tasks for users. This shows a shift towards AI that can take action, not just chat.
  3. The best AI models now cost more, with some requiring premium subscriptions. While powerful models like GPT-4.5 have high access fees, other new features may be available for free with some limits.

HN blogs - 1/11/24

HackerNews blogs newsletter • 59 implied HN points • 02 Nov 24
🕹 Technology Software Engineering Data Analysis Cybersecurity Programming
  1. Measuring technical debt is crucial for leaders, especially CTOs. It helps in understanding and managing the challenges in software development.
  2. Freezing CEO salaries during layoffs can create a fairer work environment. It shows accountability and may protect jobs for regular employees.
  3. Life shouldn't solely be based on statistics. Everyone's experiences are unique and can't be fully represented by numbers.

ChatGPT in Shambles

Marcus on AI • 13161 implied HN points • 04 Feb 25
🕹 Technology Artificial Intelligence Machine Learning Natural Language Processing Data Analysis Software Development
  1. ChatGPT still has major reliability issues, often providing incomplete or incorrect information, like missing U.S. states in tables.
  2. Despite being advanced, AI can still make basic mistakes, such as counting vowels incorrectly or misunderstanding simple tasks.
  3. Many claims about rapid progress in AI may be overstated, as even simple functions like creating tables can lead to errors.

“Nvidia could soon take a serious hit, too”

Marcus on AI • 4228 implied HN points • 27 Jan 25
🕹 Technology AI Market Trends Economics Tech Companies Data Analysis
  1. Nvidia's stock might be facing a big drop, which is a concern for investors. A decline over 10% indicates that something is going on in the market.
  2. The market can behave in unpredictable ways, and this uncertainty can be tough for investors to manage. Today might be a key moment in the stock market.
  3. Overall, the economics of generative AI can lead to unexpected changes, making it a wild area to watch for investors and tech enthusiasts.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

The Most Major Hurricanes Ever

The Honest Broker Newsletter • 2973 implied HN points • 27 Jan 25
🌞 Climate & Environment Climate change Weather Patterns Natural Disasters Data Analysis
  1. In 2024, there were a lot of major hurricanes, tying with 2015 for the highest since records began, which raises questions about climate patterns.
  2. Despite the increase in hurricane landfalls, there hasn't been a clear trend showing that hurricanes are becoming more intense or frequent over time.
  3. Experts believe that while human activity may influence hurricanes, detecting these changes amidst natural variability is very challenging.

A use-theory of testing

arg min • 436 implied HN points • 24 Oct 24
🔬 Science Statistics Research Mathematics Data Analysis Theory
  1. Statistical tests are designed to help separate real signals from random noise. It's not just about understanding what they mean, but what they can do in practical situations.
  2. Many people misuse statistical tests, which can lead to misunderstandings about their purpose. Communities should establish clear guidelines on how to use these tests correctly.
  3. The main function of statistical tests is to regulate opinions and decisions in various fields like tech and medicine. They help ensure that important standards are met, rather than just preventing errors.

Your Lab Tests

Ground Truths • 15921 implied HN points • 14 Dec 24
🏥 Health & Wellness Lab Tests Data Analysis
  1. Your individual lab results, like the Complete Blood Count (CBC), can vary a lot between people but stay stable for you over time. This means your personal health data can give more accurate insights than just average values used for everyone.
  2. Personalized reference values from CBC tests can help predict health risks better than conventional methods. They show clearer connections to potential diseases and can indicate specific health issues.
  3. Using advanced technology like AI to analyze these personal health metrics could help doctors spot risks early. This approach can enhance patient care by identifying high-risk individuals for proactive health management.

Silver Bulletin pollster ratings, 2025 update

Silver Bulletin • 373 implied HN points • 17 Feb 25
🇺🇸 U.S. Politics Polling Elections Data Analysis Political trends Political Reporting
  1. The latest pollster ratings show which pollsters are most accurate and transparent based on their past performances. This helps understand which ones might do well in future elections.
  2. New data added to the ratings includes results from the 2024 presidential, congressional, and gubernatorial elections. Lots of new polls have shifted some ratings, but the top pollsters generally stayed the same.
  3. They measure pollster accuracy using different ratings and scores that consider factors like bias toward political parties and how close their predictions were to actual results.

What Is Statistics' Purpose?

arg min • 734 implied HN points • 14 Oct 24
🔬 Science Statistics Research Medicine Philosophy Data Analysis
  1. Statistics should help us test claims by measuring how surprising the results are. However, there's doubt about whether our current statistical tests actually do this well.
  2. Randomized trials are important because they help us learn about treatments that may not always work. They focus on safety as much as they do on finding effective solutions.
  3. The field of statistics needs to be clear about its purpose. We should distinguish between using statistics for proving theories and for practical decision-making like quality control.

Measuring Programmer Influence, Kinda Sorta

Software Design: Tidy First? • 1347 implied HN points • 27 Jan 25
🕹 Technology Software Development Data Analysis Programming Project management
  1. Data can provide hints about a programmer's influence, but it can't give a clear answer. It's important to interpret the data with caution and avoid making strict decisions based solely on it.
  2. Creating files is one way to measure initiation of influence, but it's not the only factor. The impact is also determined by how frequently those files are modified by others.
  3. Using data for bonuses or promotions can lead to problems. It's better to focus on improvement and impact rather than just the numbers, to maintain a healthy team dynamic.

How did the polls do in 2024? It’s complicated.

Silver Bulletin • 312 implied HN points • 17 Feb 25
🇺🇸 U.S. Politics Polling Elections Public Opinion Data Analysis Political Bias
  1. Polls in 2024 had a lower average error than in previous years, which shows improvement in their accuracy. However, most polls underestimated Republican candidates, particularly Trump.
  2. There was a consistent bias in polls, leaning towards Democrats over the past three elections. This trend is concerning as it suggests a systematic issue with polling methods.
  3. Polling accuracy in calling election winners was lower in 2024 compared to past years. Close races should be seen as uncertain, and small leads in polls don't mean much.

The Shape of Stats to Come

arg min • 634 implied HN points • 10 Oct 24
🚌 Education Statistics Optimization Philosophy Data Analysis Mathematics
  1. Statistics often involves optimizing methods to get the best results. Many statistical techniques can actually be viewed as optimization problems.
  2. Choosing a statistical method isn't just about the math—it's also based on beliefs about reality. This philosophical side is important but often overlooked.
  3. There's a danger in relying too much on tools and models we can solve. Sometimes, we force the data to fit our preferred methods instead of being open to the actual complexities.

Roche Nanopore: Accuracy

ASeq Newsletter • 7 implied HN points • 28 Feb 25
🔬 Science Biotechnology Genomics Data Analysis Medical Research
  1. Roche's Q39 accuracy system is different from other platforms like Illumina and Oxford Nanopore. It's important to compare them carefully as each has unique metrics.
  2. The average accuracy of different sequencing platforms varies, but Roche doesn't provide clear comparisons. They share limited data about their simplex accuracy.
  3. Understanding the differences in data quality and error rates across platforms is crucial. Factors like read length and error filtering play a significant role in the accuracy of sequencing results.

AI analysis: ChatGPT vs Claude

Handy AI • 19 implied HN points • 29 Oct 24
🕹 Technology AI Data Analysis Machine Learning Software Development Information Technology
  1. ChatGPT performed better in analyzing a Spotify dataset, providing accurate insights without errors, and displaying clear visualizations.
  2. Claude encountered issues with text extraction and made mistakes in data interpretation, like incorrectly assigning genre labels where they didn't exist in the dataset.
  3. Overall, ChatGPT offered a smoother user experience, allowing users to follow along with the analysis while Claude's process was less straightforward.

The new AI scaling law shell game

Marcus on AI • 4663 implied HN points • 24 Nov 24
🕹 Technology Artificial Intelligence Machine Learning Computing Data Analysis
  1. Scaling laws in AI aren't as reliable as people once thought. They're more like general ideas that can change, rather than hard rules.
  2. The new approach to scaling, which focuses on how long you train a model, can be costly and doesn't always work better for all problems.
  3. Instead of just trying to make existing models bigger or longer-lasting, the field needs fresh ideas and innovations to improve AI.

The Political Preferences of DeepSeek AI Models

Rozado’s Visual Analytics • 283 implied HN points • 29 Jan 25
🇺🇸 U.S. Politics Political Science AI Ethics Public Policy Data Analysis Technology Trends
  1. DeepSeek AI models show political preferences similar to those of American models. This suggests that AI might reflect human biases in their programming.
  2. The findings indicate that AI can carry the same ideologies as the people who create and train them. It's important to be aware of this influence.
  3. For those curious about how political preferences impact large language models, there are more detailed analyses available to explore.

Can Your AI Strategy Be Future-Proof? | Galileo’s Vikram Chatterji

Dev Interrupted • 18 implied HN points • 18 Feb 25
🕹 Technology AI Engineering Data Analysis Productivity
  1. AI models sometimes miss important details, like humans do. For example, they may overlook obvious outliers in data visualizations.
  2. Banks are changing their hiring tactics to attract tech talent by offering more flexibility and modern tools. This helps them stay competitive against tech firms.
  3. In a world where AI is growing, the ability to focus deeply is becoming more valuable than just knowing how to use AI tools. Staying focused can help engineers excel.

DeepSeek-V3: Technical Details

Gonzo ML • 252 implied HN points • 06 Feb 25
🕹 Technology Artificial Intelligence Machine Learning Computer Science Data Analysis Software Development
  1. DeepSeek-V3 uses a new technique called Multi-head Latent Attention, which helps to save memory and speed up processing by compressing data more efficiently. This means it can handle larger datasets faster.
  2. The model incorporates an innovative approach called Multi-Token Prediction, allowing it to predict multiple tokens at once. This can improve its understanding of context and boost overall performance.
  3. DeepSeek-V3 is trained using advanced hardware and new training techniques, including utilizing FP8 precision. This helps in reducing costs and increasing efficiency while still maintaining model quality.

SQL and Python Combined Change RevOps

beyondrevenueoperations • 19 implied HN points • 27 Oct 24
💼 Business Data Analysis Automation Technology Integration Strategic Planning
  1. Combining SQL and Python makes data management much easier. SQL helps you access and pull data, while Python helps analyze it and create reports.
  2. Using SQL, you can break down data silos from different systems to get a complete view of your customers and performance. This is crucial for making smart, data-driven decisions.
  3. With Python, you can automate tasks, build predictive models, and visualize data, which saves time and enhances your ability to understand trends and insights.

5 Key Predictions for the Data Industry in 2025

SeattleDataGuy’s Newsletter • 612 implied HN points • 07 Jan 25
🕹 Technology Data Analysis AI Data Tools Business Intelligence Data Governance
  1. Iceberg will become popular, but not every business will adopt it. Many companies want simpler solutions that fit their needs without needing lots of complicated tools.
  2. SQL isn't going anywhere; it still works well for managing and querying data. People have realized that a bit of order in data is important for getting meaningful insights.
  3. AI use will become more practical, focusing on real-world applications rather than just hype. Companies will find specific tasks to automate using AI, making their workflows more efficient.

Do OpenAI's New Reasoning Models (o1 Series) Differ Politically from Their Predecessors?

Rozado’s Visual Analytics • 150 implied HN points • 28 Jan 25
🕹 Technology AI Models Data Analysis Reasoning Political Bias Model development
  1. OpenAI's new o1 models are designed to solve problems better by thinking through their answers first. However, they are much slower and cost more to run than previous models.
  2. The political preferences of these new models are similar to earlier versions, despite the new reasoning abilities. This means they still lean left when answering political questions.
  3. Even with their advanced reasoning, these models didn't change their political views, which leads to questions about how reasoning and political bias work together in AI.

China’s DeepSeek Adds a Weird New Data Point to The AI Race

Am I Stronger Yet? • 282 implied HN points • 30 Jan 25
🕹 Technology AI Models Machine Learning Data Analysis AI Research Competitor Analysis
  1. DeepSeek's new AI model, r1, shows impressive reasoning abilities, challenging larger competitors despite its smaller budget and team. It proves that smaller companies can contribute significantly to AI advancements.
  2. The cost of training r1 was much lower than similar models, potentially signaling a shift in how AI models might be developed and run in the future. This could allow more organizations to participate in AI development without needing huge budgets.
  3. DeepSeek's approach, including releasing its model weights for public use, opens up the possibility for further research and innovation. This could change the landscape of AI by making powerful tools more accessible to everyone.

What Homes Are Missing, again?

Erdmann Housing Tracker • 231 implied HN points • 03 Feb 25
💼 Business Housing Economics Data Analysis Market Trends Population Growth
  1. There is a significant shortage of homes in the U.S., estimated at around 15 million. This is due to various factors like vacancies and the rising number of adults per home.
  2. Vacancies have dropped over the years, and we might be short about 5 million vacant units needed to keep rent inflation stable.
  3. Population growth has slowed since 2008 and has likely affected housing demand, which adds pressure to the existing housing shortage.

When will psychiatry invent the barcode?

Wood From Eden • 1344 implied HN points • 04 Dec 24
🏥 Health Politics Mental health Psychiatry Diagnosis Research Data Analysis
  1. Psychiatry has a problem with labels. Many old labels have been removed without clear replacements, making research and understanding harder.
  2. Using numbers instead of words could help describe a person's mental health better. A barcode-like system could show traits and abilities at a glance.
  3. Psychology is subjective and changes over time. Collecting more data through tests can help improve understanding and research in mental health.

How many people died in disasters in 2024?

Sustainability by numbers • 211 implied HN points • 27 Jan 25
🌞 Climate & Environment Disasters Sustainability Data Analysis Public Health Environmental Impact
  1. In 2024, fewer people died from disasters compared to previous years, thanks to fewer major earthquakes. The estimate was around 9,500 deaths, which is low compared to the high averages from past years.
  2. Floods, wildfires, and storms were the main causes of deaths in 2024. Many fatalities came from extreme weather events, particularly flooding in Africa and wildfires in South America.
  3. It's important to note that data on disaster deaths is often incomplete, especially for temperature-related deaths. Researchers have to estimate these numbers, leading to less reliable statistics overall.

COVID time series graphs show clearly the COVID vaccine kill people. That's why they keep the plots hidden from view.

Steve Kirsch's newsletter • 9 implied HN points • 11 Jun 25
🏥 Health Politics Public Health Vaccines Safety Data Analysis Medical Ethics
  1. Time series graphs can show if a vaccine is safe or not by plotting daily deaths after vaccination. A safe vaccine should show a flat line after the initial period.
  2. Current data for COVID vaccines shows increasing mortality rates after vaccination, which suggests they may not be safe. Many reports don’t show this data.
  3. The medical community often ignores clear signs of vaccine risks, despite evidence appearing in graphs and reports, leading to frustration among those who analyze the data.

Background Work

Software Design: Tidy First? • 1568 implied HN points • 28 Oct 24
🕹 Technology Software Design Programming Engineering Data Analysis Artificial Intelligence
  1. Background work is doing extra research or tasks beyond what's necessary. It's a way to learn and grow your skills.
  2. Successful programmers often engage in background work, which helps them become more knowledgeable and credible.
  3. While background work can sometimes feel like extra effort, it usually pays off quickly and can save time in the long run.

Thinking Like an Owner: Elevating Your Data Team's Impact In 2025

SeattleDataGuy’s Newsletter • 788 implied HN points • 30 Nov 24
💼 Business Data Analysis Team Management Career growth Performance Metrics
  1. Data teams should focus on projects that really matter to the business, not just completing tasks. It's important to pick work that makes a difference.
  2. Understanding how your business works is key to finding valuable projects. Ask questions about the data to see what's impacting your important metrics.
  3. Shift your mindset from being a regular team member to thinking like a business owner. This means taking initiative and seeking out projects that align with overall business goals.

Thank You for 100k: A Journey Through 2024 and Beyond

SeattleDataGuy’s Newsletter • 294 implied HN points • 31 Dec 24
💼 Business Consulting Data Analysis Content creation Leadership Community Building
  1. In 2024, I gained over 100,000 subscribers on both YouTube and Substack. I really appreciate the support and plan to create even better content next year.
  2. This year showed trends like cloud data migrations and smaller, fractional data teams, which are changing how companies handle data. It's important to keep an eye on these shifts in the data world.
  3. Looking ahead to 2025, I want to finish my book on data leadership and offer more webinars and mini-courses. I'm excited to engage even more with my readers and build a community.

A Pfizer shot. A failed heart. A transplant. Get vaxxed again?

RESCUE with Michael Capuzzo • 9787 implied HN points • 08 Jun 23
🏥 Health Politics Vaccination Data Analysis
  1. John Berndsen's heart complications after receiving the Pfizer vaccine illustrate a potential link to myocarditis and the importance of questioning vaccine safety.
  2. Many adverse reactions to COVID-19 vaccines are not being reported in the media, and the numbers show a significant impact on health, including deaths.
  3. John Berndsen's experience highlights the importance of critically examining the safety and necessity of additional vaccine doses, especially for vulnerable individuals.

AI SOC Automation isn't the right problem to solve

Frankly Speaking • 152 implied HN points • 14 Jan 25
🕹 Technology AI Cybersecurity Information Security Data Analysis Automation
  1. Focusing on better detection engineering is key in security operations. It helps identify threats more effectively rather than just automating processes.
  2. Many traditional security operations centers (SOCs) may not be necessary for most companies. Smaller, more efficient models or managed detection services can be better alternatives.
  3. The future of SOCs is likely to involve fewer human analysts and more automation, emphasizing custom detections that fit the specific needs of a business.

Polling is becoming more of an art than a science

Silver Bulletin • 214 implied HN points • 16 Jan 25
🇺🇸 U.S. Politics Polling Political trends Voter Behavior Data Analysis
  1. Polling accuracy is becoming less predictable and more nuanced. Pollsters are feeling cautiously optimistic this time, although mistakes still happened in predicting election outcomes.
  2. Pollsters are likely to stick with their current methods for 2026. Many have already adapted and believe the changes they've made are effective enough for now.
  3. There is no single best way to conduct polls anymore. Different methods and tech are used by different polling organizations, which can lead to varied results.

TSMC v Intel in Two Charts

Tim Culpan’s Position • 119 implied HN points • 05 Sep 24
🕹 Technology Semiconductors Charts Data Analysis Market Trends
  1. TSMC and Intel are two major players in the semiconductor industry. Their performance and strategies have crucial implications for technology.
  2. Visual data can highlight important differences in the technical and financial health of these companies. Charts can make complex information easier to understand.
  3. Recent reports show that Intel is facing significant challenges, while TSMC continues to lead in production and technology advancements. This could shape the future of the tech industry.

Deconstructing the Transformers ReAct JSON System Prompt

Encyclopedia Autonomica • 39 implied HN points • 13 Oct 24
🕹 Technology AI Software Machine Learning Data Analysis Development
  1. Transformers use a specific structure for commands called JSON. This makes it easier to describe actions clearly and effectively.
  2. The system prompt includes rules that the agent must follow, like focusing on one action at a time and using the correct values for inputs.
  3. The design also emphasizes iterative reasoning, where the agent can build on previous observations to make better decisions in tasks.

Conservatives Are Lying on Immigrant Crime

Richard Hanania's Newsletter • 3657 implied HN points • 07 Oct 24
🇺🇸 U.S. Politics Immigration Policy Crime Rates Public Discourse Political Rhetoric Data Analysis
  1. Many people incorrectly believe that immigration leads to higher crime rates. In reality, data shows that most immigrants, especially legal ones, tend to commit less crime than native-born citizens.
  2. Some politicians use scary language about immigrants increasing crime to push their agenda. This can create a false narrative that makes the public fearful and misinformed about the actual impact of immigration.
  3. Immigrants often face more crime themselves and can actually help reduce crime rates in communities by starting businesses and contributing to the economy. So, they can serve as a buffer against crime rather than a cause of it.

Top Links 575 Global chip chain. Sino-Saudi contacts. The Neolithic economic revolution. Data driven.

Chartbook • 486 implied HN points • 30 Nov 24
🕹 Technology Global Trade Economic Systems Data Analysis
  1. The global chip supply chain is crucial for technology and industry. It's important for countries to manage and protect these resources.
  2. Sino-Saudi relations are growing, showing a shift in geopolitical alliances. Countries are forming partnerships based on mutual interests.
  3. The Neolithic economic revolution marked a significant change in human society. It reminds us how major shifts can change how we live and work.

Is being data driven a fad?

benn.substack • 843 implied HN points • 18 Oct 24
💼 Business Data Analysis Financial Metrics Market Trends Investment Strategies Business Models
  1. The way we value companies might be changing. Instead of just looking at numbers, people are considering things like hype and public interest.
  2. Being data-driven used to be seen as a key to success, but now it seems less effective for some businesses. There are successful examples, but many companies struggle to use data well.
  3. Cultural factors, or 'taste', are becoming more important in the business world than just relying on data. This shift might mean that how people feel about a company matters just as much as the finances.

Who Predicted 2023?

Astral Codex Ten • 8534 implied HN points • 05 Mar 24
🕹 Technology Forecasting Prediction Contest Algorithm Data Analysis
  1. The Annual Forecasting Contest on astralcodexten.com involves participants making predictions about various questions, helping to determine if one identifiable genius or aggregated mathematical predictions work best for foreseeing the future.
  2. The winners of the contest were both amateurs and seasoned forecasting veterans, showcasing a mix of skill and luck in predicting outcomes.
  3. Metaculus outperformed prediction markets, superforecasters, and the wisdom of crowds in the contest, suggesting that consistent high performance might be rare but achievable with specific methods like those used by superforecaster Ezra Karger.