The hottest Statistics Substack posts right now

And their main takeaways
Category
Top Technology Topics

Social Scientists Are Lazy

Cremieux Recueil • 477 implied HN points • 25 Mar 26
🔬 Science Statistics
  1. Researchers often use between-person comparisons that aren’t causally informative even when within-person or sibling designs are possible, so their estimates can be biased by unmeasured confounders.
  2. When you run within-family or within-person analyses, many headline associations (for example, claims that more social media use lowers cognition) disappear, suggesting those original results were artifacts of confounding.
  3. The field routinely skips basic robustness checks and measurement-invariance tests; empowering methodologists, providing better tools, and enforcing stricter editorial standards would greatly improve research reliability.

I swear the UFO is coming any minute

Experimental History • 21198 implied HN points • 17 Feb 26
🔬 Science Statistics
  1. Many famous psychology and neuroscience findings are under fresh scrutiny because of shady methods, tiny samples, or failed replications, so canonical stories aren’t as solid as they once seemed.
  2. How researchers measure things matters a lot — using correlation versus absolute error can lead to opposite conclusions about whether people understand how public opinion has changed.
  3. A bunch of curious, practical items matter too: interviews, art and career advice, puzzles and internet myths show the value of digging deeper, and a few vocal individuals often dominate complaint systems and waste resources.

Stealing Signals, Week 8, Part 1

Stealing Signals • 679 implied HN points • 29 Oct 24
🎾 Sports Statistics
  1. NFL scoring has improved this season, with teams averaging 22.7 points per game, up from the previous years where numbers were lower. This shift shows offenses taking more risks and being more productive.
  2. The tight end position has seen a resurgence, with many players starting to score well again. Guys like Kyle Pitts and George Kittle are performing better, and more TEs are averaging double-digit PPR points.
  3. Certain teams are now focusing more on their run game, which affects how much they pass. The Eagles, for example, have been running more, leading to fewer pass attempts for their key receivers, though that trend might not last.

Record Low Crime Rates Are Real, Not Just Reporting Bias Or Improved Medical Care

Astral Codex Ten • 25534 implied HN points • 18 Feb 26
🇺🇸 U.S. Politics Statistics
  1. U.S. violent and property crime rates are at or near historic lows, with the murder rate possibly the lowest in 250 years and many crimes at multi-decade lows.
  2. The decline looks real rather than just underreporting, because independent victim surveys, consistently reported crimes like car theft, and murder counts all show similar downward trends.
  3. Improved medical care doesn’t explain the drop in murders—lethality per violent incident has stayed stable or injuries have grown worse—and researchers offer multiple plausible explanations (technology, policing, demographics, lead decline, etc.) without a single agreed cause.

What If We Looked At All Of Questionspace?

Knowingless • 1566 implied HN points • 12 Mar 26
🔬 Science Statistics
  1. Scales are groups of survey items found with factor analysis that let you measure hidden traits efficiently, but they need lots of questions and many respondents to be reliable, and metrics like Cronbach’s alpha can be gamed by redundant items.
  2. Which items you include strongly shapes what factors you find, so a narrow or biased question set will miss whole traits; crowdsourcing a huge swath of questions can reveal unexpected dimensions but doesn’t eliminate sampling or submission bias.
  3. When you open up question-space widely, the biggest stable dimensions that tend to pop out are political left–right, belief/mysticism versus rationality, and a happy-versus-sad emotional axis, with many smaller subfactors depending on how finely you break the data.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

The Higgs Discovery Did Not Take Place

arg min • 1071 implied HN points • 22 Oct 24
🔬 Science Statistics
  1. The Higgs boson was theoretically discovered, but many people argue that this claim isn't solid due to complex statistical methods used in the research. It's not just about finding a particle; it's heavily based on probabilities.
  2. A lot of the processes in particle physics rely on trust within scientific communities and committees. They decide what counts as 'discovery' often through agreed conventions rather than direct proof.
  3. Questions about the Higgs boson reflect broader concerns in science regarding accountability. It shows that scientific findings often come down to people, their processes, and their decisions rather than just raw data.

Input Volatility, Week 8

Stealing Signals • 439 implied HN points • 25 Oct 24
🎾 Sports Statistics
  1. In leagues with WR/TE slots, tight ends (TEs) lose their value significantly because they are often less productive than wide receivers (WRs). It’s better to draft WRs instead of focusing on TEs in these formats.
  2. Carrying multiple quarterbacks (QBs) or kickers in fantasy football is usually unnecessary. If your league allows many players at these positions, it’s smarter to use your bench for players with more potential, instead of holding onto backups.
  3. When building your fantasy team, prioritize players with upside and avoid 'roster-clogging' players who don't help you win. It’s important to have flexible bench spots to adapt to injuries or bye weeks.

Toward a Transformative Hermeneutics of Standard Model Physics

arg min • 456 implied HN points • 25 Oct 24
🔬 Science Statistics
  1. The Higgs discovery shows how science relies on consensus rather than just statistics. It's all about how many scientists agree on something, and that's what really gives it weight.
  2. Complex governance structures are necessary in big science projects. These systems help teams work together and make important decisions about groundbreaking discoveries.
  3. Sometimes, playful writing can lead to misunderstandings. It's important to find the right balance between being engaging and being precise when discussing complex topics.

A use-theory of testing

arg min • 436 implied HN points • 24 Oct 24
🔬 Science Statistics
  1. Statistical tests are designed to help separate real signals from random noise. It's not just about understanding what they mean, but what they can do in practical situations.
  2. Many people misuse statistical tests, which can lead to misunderstandings about their purpose. Communities should establish clear guidelines on how to use these tests correctly.
  3. The main function of statistical tests is to regulate opinions and decisions in various fields like tech and medicine. They help ensure that important standards are met, rather than just preventing errors.

COOPER NCAA basketball ratings

Silver Bulletin • 149 implied HN points • 18 Mar 26
🎾 Sports Statistics
  1. COOPER is a new power-rating system that ranks all 365 Division I men’s basketball teams using wins and losses, margin of victory, team tempo, preseason polls, and conference strength.
  2. The ratings are Bayesian/Elo-like and update continuously: an impact factor weights recent games, close matchups, conference games, and NCAA tournament games more heavily, and ratings partly carry over between seasons.
  3. COOPER offers detailed outputs (offensive/defensive ratings, strength of schedule, home-court factors), an objective-only variant, historical season-ending ratings back to 1950, and tools to convert ratings to win probabilities; tournament forecasts blend COOPER with other models and injury data and are available to subscribers.

Last Man Standing?

Knicks Film School • 853 implied HN points • 18 Oct 24
🎾 Sports Statistics
  1. The Knicks are facing uncertainty about their final player rotation spot just days before the season starts. There are many candidates, and this could change quickly, showing the team's depth challenges.
  2. Despite potential worries about the last roster spot, the team's starting lineup is strong. It might not matter much if the starters stay healthy, similar to last season's Celtics experience.
  3. Other teams are also struggling with roster questions as salary cap rules tighten. The Knicks, like many teams, need reliable backups to step up when starters are unavailable.

The Good News Is That One Side Has Definitively Won The Missing Heritability Debate

Astral Codex Ten • 27599 implied HN points • 03 Dec 25
🔬 Science Statistics
  1. Recent research shows that most traits are influenced by genetics, but researchers still can't agree on how much. Some studies suggest up to 80% heritability, while others find it closer to 30%.
  2. The new study used advanced genetic analysis on a large number of people, capturing 88% of the heritability gap previously unexplained by genetics. However, this still leaves a significant portion unaccounted for.
  3. There's a divide in how people interpret the results: some believe this study supports the idea of many rare genetic variants influencing traits, while others think it confirms that heritability might not be very high to begin with.

The biggest kink dataset is now easy to explore!

Knowingless • 1836 implied HN points • 26 Feb 26
🎭️ Culture Statistics
  1. An interactive site lets you explore a massive fetish survey of about 960,000 people and ~900 questions by picking x/y axes, filtering by demographics, and choosing weighted or unweighted views.
  2. The site includes a search, a question generator, tools to show random or statistically significant correlations, and a summary that displays exact survey wording, with some chart types still being improved.
  3. Early explorations already surface notable patterns—age-linked trends, apparent gender confounds in reported partner counts, low neuroticism predicting enjoyment of sex work, and subs reporting more interest in violent porn—so it can help people find new, testable correlations.

What Is Statistics' Purpose?

arg min • 734 implied HN points • 14 Oct 24
🔬 Science Statistics
  1. Statistics should help us test claims by measuring how surprising the results are. However, there's doubt about whether our current statistical tests actually do this well.
  2. Randomized trials are important because they help us learn about treatments that may not always work. They focus on safety as much as they do on finding effective solutions.
  3. The field of statistics needs to be clear about its purpose. We should distinguish between using statistics for proving theories and for practical decision-making like quality control.

One Weird Trick to Get Significant Results

Cremieux Recueil • 295 implied HN points • 13 Mar 26
🔬 Science Statistics
  1. Researchers often split samples and hunt for subgroups where effects become significant, but reporting subgroup "wins" without testing interactions or accounting for low power produces misleading, likely fluke results.
  2. The functional medicine trial example shows clear red flags: inconsistent numbers, bad or post-hoc preregistration, incorrect power/sample-size math, undisclosed conflicts, non-ITT analyses, and unreported/misused subgroup tests with weak measures.
  3. These practices make findings fragile and hard to replicate, so studies need proper prospective registration, correct power calculations, transparent reporting (including interaction tests), multiple-comparisons control, and shared data to be trustworthy.

The Shape of Stats to Come

arg min • 634 implied HN points • 10 Oct 24
🚌 Education Statistics
  1. Statistics often involves optimizing methods to get the best results. Many statistical techniques can actually be viewed as optimization problems.
  2. Choosing a statistical method isn't just about the math—it's also based on beliefs about reality. This philosophical side is important but often overlooked.
  3. There's a danger in relying too much on tools and models we can solve. Sometimes, we force the data to fit our preferred methods instead of being open to the actual complexities.

Designed Interactions

arg min • 257 implied HN points • 15 Oct 24
🚌 Education Statistics
  1. Experiment design is about choosing the right measurements to get useful data while reducing errors. It's important in various fields, including medical imaging and randomized trials.
  2. Statistics play a big role in how we analyze and improve measurement processes. They help us understand the noise in our data and guide us in making our experiments more reliable.
  3. Optimization is all about finding the best way to minimize errors in our designs. It's a practical approach rather than just seeking perfection, and we need to accept that some questions might remain unanswered.

Convex Optimization at the Midpoint

arg min • 198 implied HN points • 17 Oct 24
🚌 Education Statistics
  1. Modeling is really important in optimization classes. It's better to teach students how to set up real problems instead of just focusing on abstract theories.
  2. Introducing programming assignments earlier can help students understand optimization better. Using tools like cvxpy can make solving problems easier without needing to know all the underlying algorithms.
  3. Convex optimization is heavily used in statistics, but there's not much focus on control systems. Adding a section on control applications could help connect optimization with current interests in machine learning.

TPRR check-in: Who has made strides earning volume so far?

Stealing Signals • 599 implied HN points • 03 Oct 24
🎾 Sports Statistics
  1. Routes data is really important for understanding how well players are performing. Different sources measure these routes in different ways, which can create confusion.
  2. The NFL has started providing its own routes data, which could help standardize how we analyze player performance. This might make comparisons easier and clearer moving forward.
  3. Stats like TPRR (Targets Per Route Run) help us understand player efficiency, but they need to be used alongside other context like player roles and QB performance for better insights.

Inverse Problems

arg min • 515 implied HN points • 03 Oct 24
🔬 Science Statistics
  1. Inverse problems help us create images or models from measurements, like how a CT scan builds a picture of our insides using X-rays.
  2. A key part of working with inverse problems is using linear models, which means we can express our measurements and the related image or signal in straightforward mathematical terms.
  3. Choosing the right functions to handle noise and image characteristics is crucial because it guides how the algorithm makes sense of the data we collect.

The Evidence That A Million Americans Died Of COVID

Astral Codex Ten • 32210 implied HN points • 22 May 25
🏥 Health Politics Statistics
  1. Many people are unsure if the 1.2 million COVID deaths are accurate, with some believing these deaths are linked to other causes rather than COVID itself.
  2. The data shows that total deaths during the pandemic were higher than usual, which supports the idea that many deaths were directly caused by COVID.
  3. Some argue that they don’t personally know anyone who died from COVID, but with a large population, it makes sense that not everyone would know someone affected.

The One and the Many

The Fry Corner • 21522 implied HN points • 02 Feb 24
🎭️ Culture Statistics
  1. Groups of people can behave in predictable ways, even if individuals within those groups act randomly. This means we can anticipate the behavior of a crowd better than that of a single person.
  2. Statistics play a big role in predicting risks and behaviors. For example, actuarial tables help insurance companies set rates based on the likelihood of certain events, regardless of the reasons behind those probabilities.
  3. There is often a disconnect between how we view groups of people versus individuals. While we might feel negatively about humanity as a whole, we tend to appreciate and trust the individuals we meet in our daily lives.

The YouTube Thumbnail for This Would Be "POCH OUT?!?!?! FRAUD WATCH???"

No Grass in the Clouds • 159 implied HN points • 16 Oct 24
🎾 Sports Statistics
  1. Pochettino's current U.S. team is not as strong as the teams he previously coached, with a much lower market value.
  2. Key players like Pulisic and McKennie didn't fully participate in recent games, impacting the team's performance.
  3. The expected outcome of the games against Panama and Mexico matched reality, showing that even a great coach can't win with limited resources.

cats with stats: safety in the east

bad cattitude • 207 implied HN points • 21 Feb 26
🌐 International Statistics
  1. Warsaw and Poland look meaningfully safer than comparable Western European cities and countries, with a custom crime composite ranking Warsaw well below Paris, London, and several other capitals.
  2. A city’s overall immigrant share correlates with higher crime on the composite index, and that relationship is statistically significant, though Zurich is a notable outlier with high immigration but low crime.
  3. The percentage of immigrants from non‑European origins explains much more of the variation — the regression against non‑European immigrant share gives a very high R² (~0.87) and a very low p‑value (~0.0003) — but the result comes with methodological caveats and some imputed values.

The 2024 True Sack Rate (TSR): Week 7 Update

Trench Warfare • 79 implied HN points • 22 Oct 24
🎾 Sports Statistics
  1. The True Sack Rate (TSR) is a system that evaluates sacks by defensive linemen based on how they earned those sacks. It breaks down sacks into categories, helping to better evaluate a player’s skill and performance.
  2. There are four types of sacks in the TSR, ranging from rare high-quality wins over elite blockers to more common 'cleanup' sacks that are less about skill. This helps to highlight which players are truly impactful in games.
  3. Currently, top players in the TSR have recorded impressive sack scores, showcasing their effectiveness this season. Studying their performance can give fans deeper insights into who stands out on the field.

Heritability of human life span is about 50% when heritability is redefined to be something different

DYNOMIGHT INTERNET NEWSLETTER • 703 implied HN points • 05 Feb 26
🔬 Science Statistics
  1. If you measure lifespan heritability in a simulated world with no non‑aging deaths (accidents, murder, overdoses, infectious disease), the apparent heritability rises to roughly 46–57%, about 50%.
  2. Heritability is an observational ratio that depends on societal and environmental factors, so lowering extrinsic mortality naturally increases the fraction of lifespan variation attributed to genetics.
  3. The simulation is a useful exercise and matches historical twin estimates, but its strong assumptions and vague reporting mean the ~50% figure shouldn’t be taken as the true modern heritability; a more cautious read of the results suggests something closer to 35–45% (around 40%).

An Inversion Cookbook

arg min • 297 implied HN points • 04 Oct 24
🚌 Education Statistics
  1. Using modularity, we can tackle many inverse problems by turning them into convex optimization problems. This helps us use simple building blocks to solve complex issues.
  2. Linear models can be a good approximation for many situations, and if we rely on them, we can find clear solutions to our inverse problems. However, we should be aware that they don't always represent reality perfectly.
  3. Different regression techniques, like ordinary least squares and LASSO, allow us to handle noise and sparse data effectively. Tuning the right parameters can help us balance accuracy and manageability in our models.

How Your Survey Responder Lies

Knowingless • 1364 implied HN points • 15 Jan 26
🔬 Science Statistics
  1. Where and how you ask matters: public, informal polls (like Twitter) invite people to joke or troll on simple/funny questions, while private or more formal surveys tend to get more accurate answers.
  2. Some questions are especially vulnerable to ego or incentives—people give more flattering or different answers when they expect feedback or visibility (e.g., claiming to be above average or reporting horniness), but other sensitive items (like certain sexual fantasies) may not change much.
  3. There’s no one-size-fits-all rule for survey reliability; good survey design requires thinking about your audience’s incentives and visibility, testing specific questions, and adjusting phrasing or format to reduce trolling and bias.

How To Stop Worrying And Learn To Love Lynn's National IQ Estimates

Astral Codex Ten • 31522 implied HN points • 15 Jan 25
🔬 Science Statistics
  1. IQ differences between groups may not be purely genetic and can be influenced by environmental factors like nutrition and education. This means that poorer conditions in some countries can lead to lower IQ scores.
  2. People often perceive those with low IQs differently based on specific syndromes, which can cause various functional deficits. A person with a low IQ might still lead a normal life in their context.
  3. The gap in IQ scores between different groups suggests there's potential for improvement through development initiatives. Better nutrition, health care, and education can help raise IQ scores in underdeveloped areas.

Sex Differences in Intelligence

Cremieux Recueil • 295 implied HN points • 20 Feb 26
🔬 Science Statistics
  1. Average general intelligence (g) is essentially the same for men and women. Any mean gap is vanishingly small (on the order of a few tenths of an IQ point) and not practically meaningful.
  2. Men show greater variability in intelligence and test scores, producing more males at both the high and low extremes of the distribution.
  3. Most observed sex differences come from specific skills and test-level abilities (e.g., processing speed, technical knowledge, math/verbal), which appear more malleable and can change with development — for example, early female advantages often fade by adulthood.

The Other Type of Flynn Effect

Cremieux Recueil • 235 implied HN points • 23 Feb 26
🔬 Science Statistics
  1. Many reported Flynn and anti-Flynn effects are driven by measurement bias—tests change meaning across cohorts and norms get obsolete—so gains often reflect test-taking sophistication more than real changes in general ability.
  2. Some apparent cohort trends are actually sampling or compositional artifacts, for example later-born children tending to have more advantaged parents, and those apparent gains or losses often disappear in within-family (sibling) comparisons.
  3. Robust conclusions require checking measurement invariance, using within-family designs, and guarding against collinearity and low power; when those methods are applied, large population IQ shifts usually shrink or vanish.

Liverpool Without Jurgen Klopp: Still Pretty Good!

No Grass in the Clouds • 119 implied HN points • 08 Oct 24
🎾 Sports Statistics
  1. Liverpool is performing well even without their former coach Jurgen Klopp. They have adapted their game style to focus on quality shots rather than long-range attempts.
  2. The team has been effective in creating high-quality chances during matches, especially when leading. This shows a strategic approach to controlling the game.
  3. Defensively, they have given up fewer chances this season, indicating improved defensive strategies. However, they tend to concede more when holding a big lead, which may suggest a tactical choice.

True Pressure Rate (TPR): Week 6 Update

Trench Warfare • 79 implied HN points • 15 Oct 24
🎾 Sports Statistics
  1. True Pressure Rate (TPR) is a new tool for evaluating pass-rushers that focuses on the quality of pressures, not just the amount. This helps to understand who the best defenders really are.
  2. Pressures are categorized into three quality levels: Rare High Quality, High Quality, and Low Quality. This classification provides deeper insight into a player's performance and effectiveness.
  3. The Pressure Quality Ratio (PQR) compares high-quality pressures to low-quality ones. This helps identify players who may not have a lot of pressures but are still working hard and making an impact.

Are Girls Smarter Than Boys?

Cremieux Recueil • 277 implied HN points • 13 Feb 26
🔬 Science Statistics
  1. Changing test scoring to reward calibrated confidence and risk behavior instead of just right-or-wrong answers can make women appear smarter even though it measures a different thing.
  2. Including metacognitive calibration, confidence, and risk preference in an intelligence score mixes non-intelligence traits into the measure and can break the usual positive correlations across cognitive tests, producing misleading factor patterns.
  3. The correct way to compare sexes on intelligence is to use a large, diverse test battery, score accuracy normally, and compare the general intelligence factor; redefining intelligence without strong justification is not acceptable.

Gene-environment interactions: ubiquitous yet undetectable

The Infinitesimal • 359 implied HN points • 21 Aug 24
🔬 Science Statistics
  1. Gene-environment interactions (GxE) are common but hard to identify in humans. They show how genetic traits can change in different environments, affecting how we understand traits like obesity or education.
  2. There are different models to explain how genes and environments work together. Some models show that environments can amplify or change the effects of multiple genetic variants on traits.
  3. Research has found that environmental factors, like socioeconomic status or education quality, can significantly influence how genetic variations are expressed, meaning genetics alone doesn't tell the whole story about traits.

UPDATE: The truth about Target 

Popular Information • 11556 implied HN points • 31 Oct 23
🇺🇸 U.S. Politics Statistics
  1. Target closed stores due to theft, but data showed lower levels of theft in those locations.
  2. Retailers like Target may be using theft to cover up other issues affecting their businesses.
  3. The cost of organized retail crime is a small fraction of total retail losses, with operational errors and employee theft accounting for the majority of shrink.

Latest Poll Averages: 10/7

COVID Reason • 99 implied HN points • 07 Oct 24
🇺🇸 U.S. Politics Statistics
  1. Harris is leading Trump by 2.3% in the latest average polls, with 49.1% support compared to Trump's 46.8%.
  2. The data covers a date range from September 19 to October 4, showing recent polling trends.
  3. Individual state polls are available, which can provide a clearer picture of local support for the candidates.

Pit Bulls Part I: Identification

Cremieux Recueil • 567 implied HN points • 16 Jan 26
🔬 Science Statistics
  1. Pit bulls are a recognizable type of dog with a consistent, stout muscular build and behavioral traits like high gameness and persistence that come from their bull‑and‑terrier fighting ancestry.
  2. People can reliably identify pit bull–type dogs by sight; studies and large public classification tests show high accuracy, and accuracy rises as pit ancestry increases.
  3. Arguments that pit bulls can’t be identified or that they were bred to be non‑aggressive toward humans are unsupported, and common patterns of misclassification tend to hide or downplay—rather than inflate—the elevated risks tied to pit bull type dogs.

Why I am not a polygenic score nihilist

Wyclif's Dust • 5365 implied HN points • 01 Jul 25
🔬 Science Statistics
  1. Polygenic scores can explain significant aspects of outcomes like education, despite having low R-squared values. This means they can still be useful even if they don't account for everything.
  2. The effects of genetics on educational attainment can be large, showing that having a higher polygenic score can significantly increase the chances of going to university.
  3. It's important not to dismiss polygenic scores just because they have low explanatory power. They can have real, substantial effects that matter for understanding outcomes.