The hottest Data Analysis Substack posts right now

And their main takeaways
Category
Top Technology Topics

Who Predicted 2023?

Astral Codex Ten • 8534 implied HN points • 05 Mar 24
🕹 Technology Forecasting Prediction Contest Algorithm Data Analysis
  1. The Annual Forecasting Contest on astralcodexten.com involves participants making predictions about various questions, helping to determine if one identifiable genius or aggregated mathematical predictions work best for foreseeing the future.
  2. The winners of the contest were both amateurs and seasoned forecasting veterans, showcasing a mix of skill and luck in predicting outcomes.
  3. Metaculus outperformed prediction markets, superforecasters, and the wisdom of crowds in the contest, suggesting that consistent high performance might be rare but achievable with specific methods like those used by superforecaster Ezra Karger.

Polygenic Risk Scores: Ready for Prime Time?

Ground Truths • 3980 implied HN points • 19 Feb 24
🏥 Health & Wellness Genetics Risk Assessment Prevention Data Analysis
  1. Polygenic risk scores can provide valuable information on high genetic risk for diseases like heart disease and cancer, beyond traditional clinical risk factors.
  2. The use of polygenic risk scores is advancing thanks to efforts like the eMERGE consortium, incorporating multi-ancestry data and rigorous validation.
  3. Actionable polygenic risk scores have the potential to reduce health disparities and enhance preventive strategies in medical practice.

Avoiding Epistemological Nihilism

Richard Hanania's Newsletter • 3657 implied HN points • 12 Feb 24
🔬 Science Research Data Analysis
  1. Social scientists often resort to statistical relationships when randomized experiments are not feasible, which can lead to flawed conclusions due to selection effects and confounding variables.
  2. Flawed data is often worse than having no data at all, as it can mislead individuals into making decisions based on inaccurate information.
  3. To form reasonable opinions on social, political, and economic issues, it is essential to prioritize well-grounded ideas backed by theoretical reasoning and empirical data over blindly following data from flawed social science research.

TfL's AI Tube Station experiment is amazing and slightly terrifying

Odds and Ends of History • 2278 implied HN points • 12 Feb 24
🕹 Technology AI Surveillance Transportation Data Analysis Digital innovation
  1. AI technology, like the one used in TfL's Tube Station experiment, is rapidly changing and being implemented in various sectors.
  2. AI cameras at stations can have a wide range of uses, from enhancing security to improving passenger welfare and gathering statistical data.
  3. While AI technology offers numerous benefits, there are also concerns about privacy, surveillance, and potential misuse of the technology.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Which AI should I use? Superpowers and the State of Play

One Useful Thing • 506 implied HN points • 18 Mar 24
🕹 Technology AI Machine Learning Chatbots Data Analysis
  1. There are three main GPT-4 class AI models dominating the field currently: GPT-4, Anthropic's Claude 3 Opus, and Google's Gemini Advanced.
  2. These AI models have impressive abilities like being multimodal, allowing them to 'see' images and work across a variety of tasks.
  3. The AI industry lacks clear instructions on how to use these advanced AI models, and users are encouraged to spend time learning to leverage their potential.

LLMs shouldn’t write SQL

benn.substack • 1016 implied HN points • 23 Feb 24
🕹 Technology Data Analysis Artificial Intelligence Business Intelligence
  1. In business analysis, there are two main approaches: a structured method using known metrics and BI tools and a more creative, less structured method that involves asking unique questions and using tools like Excel, SQL, and Python.
  2. The prediction that natural language will replace SQL in data management interfaces is interesting, but the role of SQL might evolve rather than disappear completely, still being crucial for generating queries efficiently.
  3. Artificial intelligence can assist in tasks like drawing or writing formulas, but the precision and efficiency of code often make it a better choice for data analysis, despite the potential for AI advancements in building complex queries.

The longer I live, the wronger I get to be

Razib Khan's Unsupervised Learning • 674 implied HN points • 02 Mar 24
🔬 Science Genetics Evolution Data Analysis
  1. In the field of human population genetics, interesting times can lead to significant advancements and significant shifts in understanding.
  2. The concept of intelligence as influenced by single 'IQ genes' has been refuted in favor of the understanding that intelligence involves thousands of genes with small effects.
  3. Historical inaccuracies regarding the ancestry of European Jews, the dynamics of human evolution out of Africa, and the role of natural selection in human evolution have been corrected with new scientific discoveries and insights.

Nvidia Illumina Acquisition? The AI Foundry for Healthcare – The Hardware of Life

SemiAnalysis • 7576 implied HN points • 27 Sep 23
🕹 Technology Healthcare Genomics AI Compute Data Analysis
  1. Eroom's Law and Moore's Law are critical in Semiconductors and Drug Research, analyzing time, money, and output.
  2. Healthcare, a $4 trillion industry, lags behind in technological progress driven by Moore's Law.
  3. Illumina acquisition by Nvidia could bridge the gap in genomics, addressing bottlenecks and enabling full-stack healthcare solutions.

Lessons From Over 13 Million Naira Worth of Organic WhatsApp Sales

Day One • 758 implied HN points • 24 Feb 24
💼 Business Sales Marketing Content creation Data Analysis Customer Support
  1. Building trust and authority through valuable content is essential for selling products or services online
  2. Utilizing testimonials and free high-quality content can greatly persuade potential customers to make a purchase
  3. Addressing objections, providing ongoing support, and reducing buyer's remorse are key to maintaining customer satisfaction and loyalty

Weather and Climate Disasters in Europe

The Honest Broker Newsletter • 726 implied HN points • 12 Feb 24
🌞 Climate & Environment Data Analysis Trends
  1. Europe experiences significant economic losses due to weather and climate disasters, averaging about €15 billion annually.
  2. Storms and floods are the main causes of losses in Europe, with heatwaves also impacting the region.
  3. Data collection on disaster impacts in Europe is lacking, making it challenging to assess long-term trends in weather and climate-related losses.

A Pfizer shot. A failed heart. A transplant. Get vaxxed again?

RESCUE with Michael Capuzzo • 9787 implied HN points • 08 Jun 23
🏥 Health Politics Vaccination Data Analysis
  1. John Berndsen's heart complications after receiving the Pfizer vaccine illustrate a potential link to myocarditis and the importance of questioning vaccine safety.
  2. Many adverse reactions to COVID-19 vaccines are not being reported in the media, and the numbers show a significant impact on health, including deaths.
  3. John Berndsen's experience highlights the importance of critically examining the safety and necessity of additional vaccine doses, especially for vulnerable individuals.

8 Forecasts & Implications for the Years Ahead :: 2024+

Implications, by Scott Belsky • 1356 implied HN points • 04 Jan 24
🕹 Technology AI Education Entertainment Organizational Design Data Analysis
  1. The future will be personalized to your preferences, with digital experiences tailored to you.
  2. Local OS-native AI models will improve everyday life and redefine consumer AI, focusing on personalization, trust, and privacy.
  3. Small brands will become more competitive with big brands, AI will influence purchase decisions, and education will undergo a significant transformation.

The Youth Mental Health Crisis is International, Unless You Rely on a Flawed International Dataset

After Babel • 1118 implied HN points • 03 Jan 24
🏥 Health & Wellness Mental health Data Analysis Misinformation
  1. Researchers should stop using the Global Burden of Disease study for analyzing mental health trends.
  2. The youth mental health crisis is not just limited to America, but is an international issue in many Western countries with high levels of smartphone adoption.
  3. The Global Burden of Disease study underestimates changes in mental health statistics since 2010, especially in depression, self-harm, and suicide rates.

5 Cybersecurity Predictions for 2024

Frankly Speaking • 305 implied HN points • 29 Feb 24
🕹 Technology Cybersecurity Artificial Intelligence Data Analysis
  1. Security companies are shifting focus to platforms, leading to acquisitions and consolidations to improve operational efficiency.
  2. Cybersecurity is moving towards more building and software engineering, away from solely relying on buying tools to solve problems.
  3. The adoption of reasonable metrics is becoming crucial for cybersecurity, allowing for better justification of funding and overall security enhancement.

Mohamed Salah is irreplaceable. Let's use data to replace him, anyway.

Grace on Football • 687 implied HN points • 23 Jan 24
🎾 Sports Football Data Analysis Tactics
  1. Mohamed Salah's exceptional performance and consistency make him irreplaceable on the field.
  2. Using data and statistical analysis can help identify potential players to fill Salah's role.
  3. To replace Salah, focus on finding players who can contribute in areas like goal-scoring, creativity, and left-footedness to maintain team balance.

DRAFT: Notes: What Is the Techno-Optimist Slant on “AI”?

Brad DeLong's Grasping Reality • 169 implied HN points • 14 Mar 24
🕹 Technology AI Machine Learning Data Analysis Tech industry Digital Transformation
  1. Very large-scale, high-dimension regression and classification analysis will be game-changing, transforming bureaucracy to algorithms with significant impacts across sectors from finance to healthcare.
  2. Natural-language interfaces to databases may be challenging to control but offer more intuitive access to vast information repositories, potentially enhancing user efficiency.
  3. Autocomplete technology provides substantial time savings for white-collar workers, illustrating the significant productivity boost modern technologies can offer.

Prompt-Based Feature Engineering Part 1: Generative AI Generates Data

nick’s datastack • 1 HN point • 24 Apr 24
🕹 Technology AI Data Analysis Machine Learning Data Engineering Data processing
  1. Generative AI can generate data, impacting workflows and pipelines significantly.
  2. Using LLMs for prompt-based feature engineering can save time and effort compared to traditional methods like manual data searching and merging.
  3. While LLMs in data pipelines may feel magical, it's important to be cautious of potential inaccuracies due to the probabilistic nature of AI outputs.

Re-Examining Opponent Breakdowns

MatchQuarters • 452 implied HN points • 05 Feb 24
🎾 Sports Data Analysis
  1. Defensive coordinators should approach game planning holistically instead of solely relying on numbers.
  2. Offenses aim to create space, while defenses work to constrain it.
  3. Simplify the process of breaking down opponents by focusing on key formations, movements, and plays to develop a comprehensive game plan.

Seattle Data Guy In 2024

SeattleDataGuy’s Newsletter • 871 implied HN points • 26 Dec 23
🕹 Technology Data Analysis Consulting Content creation Community Building
  1. Seattle Data Guy's work in 2023 involved filming videos, virtual conferences, and writing articles and newsletters.
  2. Client trends in 2023 showed shifts towards greenfield projects, solution design, marketing, and education.
  3. Popular articles in 2023 covered topics like data modeling, breaking out of tutorial hell, and essential templates for data analytics.

Daily Chartbook #393

Daily Chartbook • 183 implied HN points • 02 Mar 24
💰 Finance Investing Markets Data Analysis
  1. Daily Chartbook provides 30 charts to summarize the day's events for paid subscribers.
  2. The content on Daily Chartbook can be accessed through a subscription link on their website.
  3. To view specific posts on Daily Chartbook, individuals need to be paid subscribers.

The KQL Mysteries: Chapter 5

Rod’s Blog • 456 implied HN points • 18 Jan 24
🕹 Technology Cybersecurity Data Analysis Hacking AI Threat intelligence
  1. Jon and Sofia successfully identified and captured the teenage threat actors behind a financial breach using KQL queries and OSINT techniques.
  2. The threat actors were operating from a suburban house in Seattle, Washington, and were quickly apprehended by authorities, leading to the recovery of the funds.
  3. Despite the success, Jon remains suspicious about the involvement of the Night Princess hacker group, hinting at a potential unresolved mystery for the next chapter.

The KQL Mysteries: Chapter 6

Rod’s Blog • 416 implied HN points • 22 Jan 24
🕹 Technology Cybersecurity Data Analysis Network Security
  1. Jon discovers that the Night Princess was behind the cyber-attacks on his company, manipulating data, planting false clues, and covering her tracks.
  2. Jon uses KQL skills to investigate the Night Princess's activities by analyzing logon events and network events in the company's database.
  3. Collaboration between the Night Princess, CyberGhost, and DarkAngel in the cyber-attacks surfaces, raising questions about the Night Princess's identity and motives.

What do Americans use electricity for?

Sustainability by numbers • 75 implied HN points • 19 Mar 24
🔬 Science Energy Electricity Renewable Energy Data Analysis
  1. American households primarily use electricity for heating, cooling, and controlling humidity.
  2. Future challenges in energy demand will revolve around balancing supply and demand, particularly for temperature control like heating and cooling.
  3. Electricity consumption is dominated by heating, cooling, and humidity control in households, highlighting the importance of efficient solutions in this area.

The Future of AI Diplomacy: Can The State Department Grok AI?

ChinaTalk • 133 implied HN points • 04 Mar 24
🇺🇸 U.S. Politics Diplomacy Transparency Government National Security Data Analysis
  1. AI can enhance diplomacy by streamlining bureaucratic tasks, providing accurate data for negotiations, and improving analysis processes.
  2. Risk management in the State Department varies for different tasks: while tasks like HR and IT services can run faster to match the private sector, activities like foreign assistance and passport services require a higher burden due to their public impact.
  3. Strategic use of transparency can be a strength for the U.S. in diplomacy, as seen in the Biden administration's doctrine. Leveraging transparency internally and externally can have strategic advantages over closed societies.

Mantic Monday 4/24/23

Astral Codex Ten • 3923 implied HN points • 25 Apr 23
🕹 Technology AI Data Analysis API Funding
  1. Using AI for forecasting future world events is a growing field with potential benefits over human forecasters.
  2. Metaculus has been found to be more accurate than low-information priors and its competitor Manifold Markets, showing the potential of crowdsourcing for predictions.
  3. Exploring AI forecasting through platforms like Polymarket, Metaculus, and Manifold provides insight into trends, such as the interest in prediction markets among sci-tech celebrities.