The hottest Data Analysis Substack posts right now

And their main takeaways
Category
Top Technology Topics

Time to ChatGPT/Gemini/Claude/Grok Oneself.

The Future Does Not Fit In The Containers Of The Past • 24 implied HN points • 30 Nov 25
🕹 Technology Data Analysis
  1. Using AI tools can help you better understand yourself. You can ask it personal questions like your worth or analyze your past appraisals to get insight.
  2. Having deep conversations with other people can reveal a lot. You can ask about their most impactful experiences and compare their answers to what AI might say.
  3. It's important to think about how AI will change jobs and industries. Asking challenging questions to yourself, others, and AI can help you adapt and prepare for the future.

Online Reviews: An Untapped Resource for UX Research?

UX Psychology • 119 implied HN points • 26 Jan 24
🕹 Technology Data Analysis
  1. Online reviews offer easy access to real user feedback, going beyond predefined questions and providing insights into user profiles and product features that traditional research may miss.
  2. Large datasets from online reviews allow for analysis at a vast scale, enabling the discovery of weak signals affecting small user subsets that traditional research could overlook, especially in companies with limited research budgets.
  3. Sentiment analysis of online reviews can uncover user frustrations, needs, and pain points, helping identify where experiences fall short of expectations and providing insights into specific features and aspects of the user experience.

Data at Depth 11: Diversity, Adversity, Case Study With GPT-4 and External Validation

Data at Depth • 59 implied HN points • 18 Apr 24
🕹 Technology Data Analysis
  1. Documenting and analyzing your journey as a creator can help identify patterns of growth and areas for improvement, like diversification across social media platforms.
  2. Engaging in strategic thinking, research, and creation can lead to significant accomplishments, such as getting articles published and boosted, validating your skills as a writer.
  3. When using tools like GPT-4 for tasks like title generation, it's crucial to validate their output externally to ensure accuracy and effectiveness.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

China’s DeepSeek Adds a Weird New Data Point to The AI Race

Am I Stronger Yet? • 282 implied HN points • 30 Jan 25
🕹 Technology Data Analysis
  1. DeepSeek's new AI model, r1, shows impressive reasoning abilities, challenging larger competitors despite its smaller budget and team. It proves that smaller companies can contribute significantly to AI advancements.
  2. The cost of training r1 was much lower than similar models, potentially signaling a shift in how AI models might be developed and run in the future. This could allow more organizations to participate in AI development without needing huge budgets.
  3. DeepSeek's approach, including releasing its model weights for public use, opens up the possibility for further research and innovation. This could change the landscape of AI by making powerful tools more accessible to everyone.

Do Large Language Models have a "theory of mind"?

The Counterfactual • 219 implied HN points • 14 Sep 23
🕹 Technology Data Analysis
  1. Large language models (LLMs) show some ability to understand the beliefs of other characters in scenarios, indicating a form of Theory of Mind. This means they can predict behaviors based on what a character knows or believes.
  2. However, LLMs don't perform as well as humans on these tasks, suggesting their understanding is not as deep or reliable. They score above chance but below the typical human accuracy.
  3. Research on LLMs and Theory of Mind is ongoing, raising questions about how these models process mental states compared to humans and if traditional tests for mentalizing are sufficient.

The Sequence Radar #692: Qwen Unleashed: This Week’s Breakthrough AI Models

TheSequence • 105 implied HN points • 27 Jul 25
🕹 Technology Data Analysis
  1. Alibaba has released new AI models called Qwen that are breaking records in tasks like coding and translation. These models are designed to help developers work more efficiently.
  2. The new Qwen models include features like better reasoning and reduced memory requirements, making them accessible for more people. This means businesses can use AI without needing expensive hardware.
  3. Alibaba plans to continue expanding these models with more specialized features and improvements in understanding language and images. This shows their commitment to leading in open-source AI technology.

Top Links 537 TIGER indices looking grim. South Sudan's misery. The 1990s revisited and Soldier's Paradise.

Chartbook • 400 implied HN points • 21 Oct 24
💰 Finance Data Analysis
  1. The TIGER indices are showing a negative trend, indicating economic challenges ahead. This suggests that global economic recovery may be slower than expected.
  2. South Sudan is facing significant difficulties, highlighting ongoing humanitarian issues. These problems need urgent attention to improve the situation for its people.
  3. There are connections being made to the 1990s, suggesting that some current geopolitical situations may resemble past conflicts. This raises concerns about the repetition of history in today's world.

This Is The Most Important Thing We Can Build With AI

America 2.0 (by Gary Sheng) • 216 implied HN points • 05 Apr 23
🕹 Technology Data Analysis
  1. A human-powered, AI-supercharged network is crucial to make collective decisions and bring about positive change.
  2. The bottleneck to effective coordination lies in the quality of input data in attempts to coordinate.
  3. An AI-powered civic information network can revolutionize our ability to understand collective desires and serve the community better.

Thank You for 100k: A Journey Through 2024 and Beyond

SeattleDataGuy’s Newsletter • 294 implied HN points • 31 Dec 24
💼 Business Data Analysis
  1. In 2024, I gained over 100,000 subscribers on both YouTube and Substack. I really appreciate the support and plan to create even better content next year.
  2. This year showed trends like cloud data migrations and smaller, fractional data teams, which are changing how companies handle data. It's important to keep an eye on these shifts in the data world.
  3. Looking ahead to 2025, I want to finish my book on data leadership and offer more webinars and mini-courses. I'm excited to engage even more with my readers and build a community.

We don’t need another SQL chatbot

benn.substack • 1227 implied HN points • 14 Jul 23
🕹 Technology Data Analysis
  1. We want chatbots to handle tedious job tasks but maybe not the fun parts.
  2. Building a good text-to-SQL bot requires more than just using large language models like GPT.
  3. Technology can help us focus on creative tasks rather than just automating mechanical work.

Seattle Data Guy In 2024

SeattleDataGuy’s Newsletter • 871 implied HN points • 26 Dec 23
🕹 Technology Data Analysis
  1. Seattle Data Guy's work in 2023 involved filming videos, virtual conferences, and writing articles and newsletters.
  2. Client trends in 2023 showed shifts towards greenfield projects, solution design, marketing, and education.
  3. Popular articles in 2023 covered topics like data modeling, breaking out of tutorial hell, and essential templates for data analytics.

Work in progress on natural selection in the US

Wyclif's Dust • 1073 implied HN points • 17 Sep 23
🔬 Science Data Analysis
  1. Polygenic scores predicting education levels also predict fertility in opposite directions.
  2. Economic theory explains the relationship between income, education, and number of children.
  3. US data on natural selection shows differences compared to the UK, possibly influenced by factors like welfare support and class distinctions.

DeepSeek-V3: Technical Details

Gonzo ML • 252 implied HN points • 06 Feb 25
🕹 Technology Data Analysis
  1. DeepSeek-V3 uses a new technique called Multi-head Latent Attention, which helps to save memory and speed up processing by compressing data more efficiently. This means it can handle larger datasets faster.
  2. The model incorporates an innovative approach called Multi-Token Prediction, allowing it to predict multiple tokens at once. This can improve its understanding of context and boost overall performance.
  3. DeepSeek-V3 is trained using advanced hardware and new training techniques, including utilizing FP8 precision. This helps in reducing costs and increasing efficiency while still maintaining model quality.

Unleashing LLMs in Cybersecurity: A Playbook for All Industries

Gradient Flow • 259 implied HN points • 20 Apr 23
🕹 Technology Data Analysis
  1. Large Language Models (LLMs) are gaining interest in various industries, especially in cybersecurity, and can be used as a playbook for implementation in other domains.
  2. Custom LLMs can be created for cybersecurity applications, leading to potential advancements like specialized chatbots and content generation for enhanced security measures.
  3. LLMs are transforming automation processes in cybersecurity, offering improved accuracy and convenience, and displaying potential for impact across multiple industries through domain-specific adaptations.

Data At Depth 13: Substack Love Affair, On Reading, Python Plotly Bubbles and Bars

Data at Depth • 39 implied HN points • 16 May 24
🕹 Technology Data Analysis
  1. The author shares insights on their data analysis for the past 2 weeks, highlighting significant growth on Substack, experiences on Medium and LinkedIn, and struggles with Twitter-X.
  2. The author emphasizes the importance of taking time to read and detach from the pressure of creating content, as well as the value of ownership and direct engagement through Substack newsletters.
  3. A tutorial is provided on creating interactive Python Plotly dashboards for data visualizations, specifically focusing on a bubble map and bar chart to showcase data on global undernourishment.

Behind the scenes with '24 hours in an invisible epidemic'

Big Charts • 199 implied HN points • 29 Sep 23
🔬 Science Data Analysis
  1. The story discusses the correlation between day-to-day activities and happiness, highlighting how social interaction plays a significant role in people's well-being and happiness levels.
  2. Data visualization can sometimes present challenges in clearly conveying findings, emphasizing the importance of ensuring that the visualization aligns with the story being told.
  3. Visualizing individual diaries can make the concept of loneliness feel universal, prompting important conversations about struggles with loneliness in everyday life.

Part 1: AI Use Among UX Professionals

UX Psychology • 198 implied HN points • 17 Aug 23
🕹 Technology Data Analysis
  1. Artificial Intelligence is significantly impacting User Experience (UX) by providing new tools and methods for research and design.
  2. UX professionals have varying levels of AI knowledge and usage, with concerns including potential errors, biases, and job security.
  3. Even though many UX professionals are incorporating AI into their work, there is still caution and a desire to ensure responsible AI use and human augmentation.

Anomalous Radioactive Variations

Remote View • 196 implied HN points • 29 Mar 23
🔬 Science Data Analysis
  1. Joe Parr conducted experiments with pyramids and radioactive sources, noting cyclical variations in radioactive counts possibly linked to moon phases and solar activity.
  2. Parr's hypothesis of a 'hyperspace bubble' forming around the pyramid passing through magnetic fields is based on anomalous events in the data.
  3. The test setups involved rotating pyramids between magnetic fields, with a sophisticated setup to measure radioactive counts and variations.

Weather and Climate Disasters in Europe

The Honest Broker Newsletter • 726 implied HN points • 12 Feb 24
🌞 Climate & Environment Data Analysis
  1. Europe experiences significant economic losses due to weather and climate disasters, averaging about €15 billion annually.
  2. Storms and floods are the main causes of losses in Europe, with heatwaves also impacting the region.
  3. Data collection on disaster impacts in Europe is lacking, making it challenging to assess long-term trends in weather and climate-related losses.

The longer I live, the wronger I get to be

Razib Khan's Unsupervised Learning • 674 implied HN points • 02 Mar 24
🔬 Science Data Analysis
  1. In the field of human population genetics, interesting times can lead to significant advancements and significant shifts in understanding.
  2. The concept of intelligence as influenced by single 'IQ genes' has been refuted in favor of the understanding that intelligence involves thousands of genes with small effects.
  3. Historical inaccuracies regarding the ancestry of European Jews, the dynamics of human evolution out of Africa, and the role of natural selection in human evolution have been corrected with new scientific discoveries and insights.

Python Streamlit Spotlight: An Interactive Look At UNHCR Refugee Data

Data at Depth • 39 implied HN points • 09 May 24
🕹 Technology Data Analysis
  1. Python Streamlit is a powerful tool for creating interactive data visualizations packaged neatly into applications that can be displayed in a browser.
  2. The project highlighted step-by-step modular development to create an application with dropdown menus, radio buttons, and choropleth maps for visualizing UNHCR refugee data.
  3. The interactive Streamlit dashboard allows users to explore both where asylum seekers are going to and where asylum seekers are coming from, offering a detailed look at global refugee movements.

What Homes Are Missing, again?

Erdmann Housing Tracker • 231 implied HN points • 03 Feb 25
💼 Business Data Analysis
  1. There is a significant shortage of homes in the U.S., estimated at around 15 million. This is due to various factors like vacancies and the rising number of adults per home.
  2. Vacancies have dropped over the years, and we might be short about 5 million vacant units needed to keep rent inflation stable.
  3. Population growth has slowed since 2008 and has likely affected housing demand, which adds pressure to the existing housing shortage.

It is starting to get strange.

One Useful Thing • 1209 implied HN points • 02 May 23
🕹 Technology Data Analysis
  1. AI like GPT-4 is becoming more powerful and capable in real-world tasks
  2. Code Interpreter feature in GPT-4 allows AI to read, generate, and understand code and data autonomously
  3. Microsoft Copilot and GPT-4 plugins are revolutionizing work tasks like data analysis and document creation

Dashing Data Viz - Issue 229

Dashing Data Viz • 176 implied HN points • 14 Mar 23
🕹 Technology Data Analysis
  1. The newsletter shares articles and videos on data visualization, like creating gradient line charts in R and using Tableau for interactive dashboards.
  2. There are resources available for learning new skills in data visualization, such as an online course on Intro to R for Data Viz.
  3. The newsletter also highlights interesting projects like visualizing the first 5,000 digits of Pi and provides resources for further reading on topics like data hierarchy best practices.

Retention Analytics - the definitive guide

timo's substack • 176 implied HN points • 12 Mar 23
🕹 Technology Data Analysis
  1. Focus on retention rate, especially first-week retention for free users, as a key metric for product analytics
  2. Retention analytics require solid user identification to track if users are returning and engaging with your product
  3. Measure retention with cohorts to understand performance over time, highlighting improvements or decreases in user retention

How to Read an AI Image

Cybernetic Forests • 379 implied HN points • 02 Oct 22
🔬 Science Data Analysis
  1. AI-generated images are informative about the underlying dataset and the human decisions shaping it.
  2. When analyzing AI images, it's crucial to consider the dataset's cultural, social, economic contexts, and how they influence the output.
  3. A methodology involving creating sample sets, content analysis, database exploration, and connotative analysis can help interpret the underlying biases and limitations in AI-generated images.

How To Fast Track Your Career As An Engineer

SeattleDataGuy’s Newsletter • 930 implied HN points • 12 Aug 23
💼 Business Data Analysis
  1. Focusing on impact in your work can accelerate your career growth and lead to more satisfying outcomes.
  2. To have more impact in tech, run towards unsolved problems, be scrappy in finding solutions, and prioritize ruthlessly.
  3. Impact can be achieved by reducing costs or increasing revenue, and understanding how your work contributes to these areas is essential for career advancement in engineering.

KCOR results on the Japan record level data shows the same harm pattern as the Czech data

Steve Kirsch's newsletter • 6 implied HN points • 15 Jan 26
🏥 Health Politics Data Analysis
  1. KCOR analysis of Japan and Czech record-level data shows a consistent pattern where recently vaccinated cohorts have higher all-cause mortality than unvaccinated cohorts.
  2. The pattern appears dose-dependent, with second doses linked to higher mortality than first, and KCOR claims to avoid healthy‑vaccinee bias by using fixed enrollment cohorts and adjusting in mortality space rather than 1:1 matching.
  3. The stated conclusion is that COVID vaccines increased the net risk of death, mainstream proponents are described as unwilling to engage with the data, and an open public debate is demanded to resolve the disagreement.

A look at the Exploit Prediction Scoring System (EPSS) 3.0

Resilient Cyber • 219 implied HN points • 31 Jul 23
🕹 Technology Data Analysis
  1. EPSS 3.0 helps security teams focus on the vulnerabilities that are most likely to be exploited soon. This makes managing vulnerabilities easier and more efficient.
  2. Many organizations struggle to fix all their vulnerabilities and often end up wasting time on those that are rarely exploited. EPSS aims to change that by identifying threats more accurately.
  3. The new version of EPSS shows a big improvement in predicting which vulnerabilities are at risk. This means companies can spend less time on unimportant issues and focus on what really matters.