The hottest Research Methods Substack posts right now

And their main takeaways
Category
Top Science Topics

How to evaluate statistical claims

The Counterfactual • 199 implied HN points • 27 Jun 24
🚌 Education Research Methods
  1. Always look at the whole distribution of data, not just the average. The average can be affected by extreme values, so it's crucial to see the bigger picture to understand what the data really tells us.
  2. Consider the baseline or reference point when evaluating numbers. Knowing how a number compares to others helps us understand if it's large or small, which gives us better context.
  3. Understand the story behind the data-generating process. This means recognizing the factors that led to the results we see, which helps in identifying possible biases or alternative explanations.

How to teach science

The Bell Ringer • 219 implied HN points • 21 Jun 24
🚌 Education Research Methods
  1. Learning science should be fun, but it's important to share the complete story behind science and math. This helps students understand its real value.
  2. Teachers should balance fun activities with serious discussions about scientific concepts. This approach encourages deeper understanding.
  3. Engaging students effectively means showing them how science relates to the real world. This makes learning more relevant and interesting for them.

You Can't Just "Control" For Things

Cremieux Recueil • 803 implied HN points • 22 Jul 25
🚌 Education Research Methods
  1. Statistical controls aren’t a magic solution; using them incorrectly can lead to wrong conclusions. It's important to understand the underlying relationships between variables before just plugging numbers into an equation.
  2. Matching groups in studies to control for variables often isn't enough. You might still end up with biases if the controls aren’t comprehensive or well-measured.
  3. Over-controlling or trying to account for too many factors can confuse the results. Sometimes, less control can provide a clearer picture, just like how comparing fast food and fine dining should keep their unique qualities intact.

Psychology’s Greatest Misses (Part 2/3)

Living Fossils • 28 implied HN points • 04 Feb 26
🔬 Science Research Methods
  1. Many popular psychology claims are wrong or overstated — examples include learning-style teaching, what reaction-time implicit-bias tests prove, body-based trauma cures, and facilitated communication; believing these myths wastes time and can cause real harm.
  2. Some findings are solid but limited — the Big Five reliably describes personality differences but it describes patterns rather than explains causes and only modestly predicts specific behavior.
  3. Bad ideas spread because incentives and human storytelling favor novel, simple, or emotionally satisfying claims; novelty and neat villains travel faster than careful, boring truth, though better information tools may help correct that.
Get a weekly roundup of the best Substack posts, by hacker news affinity:

Interpreting the Turnaway Study Correctly

Bet On It • 150 implied HN points • 02 Dec 25
🏥 Health Politics Research Methods
  1. The study finds that women denied abortions report similar long-run life satisfaction to those who got abortions, suggesting the subjective benefits of parenting can offset its objective costs.
  2. Being denied an abortion still leads to real harms: prolonged financial hardship, changed life trajectories, worse relationships, health risks, and in a few cases death — emotional resilience doesn’t erase those effects.
  3. Policy claims should depend on measured magnitudes; researchers should pre-specify the effect sizes that would justify different laws, because findings that don’t change policy recommendations are misleading.

A Software Engineer's Guide to Reading Research Papers

Confessions of a Code Addict • 1467 implied HN points • 28 Jan 25
đź•ą Technology Research Methods
  1. Research papers are important for software engineers to keep up with new technologies and fill knowledge gaps. It helps to stay current with developments in your field, like time series analysis.
  2. Many people find reading research papers hard because they can be dense and technical. A lack of a research background can make it seem even more intimidating.
  3. With time and practice, anyone can learn to read and understand research papers. Finding a personal approach or framework can make the process easier.

Why Accuracy Dominates Bias and Self-Fulfilling Prophecy

Unsafe Science • 116 implied HN points • 28 Nov 25
🔬 Science Research Methods
  1. People are generally pretty accurate at judging others, and many stereotypes reflect real group differences; when people have individual information they rely on it much more than on stereotypes.
  2. Biases and self‑fulfilling prophecies do occur, but studies show their effects are typically small, fragile, and short‑lived, while the literature has often overstated their prevalence.
  3. Overemphasizing bias can lead to misguided policies and hurt the credibility of social science, so decisions should follow the full evidence and balance accuracy with non‑discrimination.

We Should Try to Directly Measure the Value of Scientific Papers

Nonsense on Stilts • 59 implied HN points • 20 Jul 24
🔬 Science Research Methods
  1. We should measure the value of scientific papers to understand their real impact. If a paper doesn't change how people act or think, then it may not be worth much.
  2. To figure out the value of a paper, we can use a formula that compares what outcomes we expect with the information from the paper versus without it. This helps us see if the research is actually useful.
  3. It's important to have good estimates and decisions tied to the research to see its true worth. By doing this, we can better judge which scientific papers are really making a difference.

Thanks But I'll Walk, Scotty

Everything Is Amazing • 1031 implied HN points • 03 Feb 25
🔬 Science Research Methods
  1. Animals, like wild mice and rats, have been observed using wheels just for fun, without any obvious benefit. This suggests that play and enjoyment are important to all creatures, not just humans.
  2. Our brains can be easily tricked by illusions like pareidolia, where we see faces in random objects. This highlights how our perception can change quickly based on how we look at things.
  3. Having fun should be a priority in how we interact with each other. Embracing joy can improve our lives and connections, just like it does for animals.

Machine Readable

Adjacent Possible • 474 implied HN points • 25 Jun 25
đź•ą Technology Research Methods
  1. Language models like AI can help researchers by making it easier to analyze and write about history. They serve as a tool to explore new ideas and angles in research.
  2. Using AI as a collaborator can enhance creativity in writing. It allows writers to experiment with different structures and topics without fully outsourcing their thoughts and decisions.
  3. While AI is helpful for summarizing and generating connections, it should not replace deep reading of primary sources. Engaging with the text is crucial for true understanding and insight.

A Chatbot Coined A Phrase I Really Like. Am I Allowed To Use It?

Singal-Minded • 470 implied HN points • 17 Jun 25
đź•ą Technology Research Methods
  1. AI can help generate new ideas and phrases that may not have been used before. Sometimes, the phrases created by AI can resonate and feel relevant in discussions.
  2. Using phrases created by AI raises questions about ownership and credit. Writers might wonder if they can use these phrases without considering who actually came up with them.
  3. The phrase 'confirmatory research theater' highlights an important issue in research, where studies might look rigorous but really just confirm what researchers wanted to prove all along.

Defining Interpersonal Trust

Heterodox STEM • 71 implied HN points • 11 Dec 25
🔬 Science Research Methods
  1. Leading researchers treat the World Values Survey question “most people can be trusted” as a measure of interpersonal trust, not just trust in people you personally know.
  2. Factor analyses show that this question loads with trust in strangers rather than with trust in friends and family, so it captures a generalized form of interpersonal trust toward unfamiliar people.
  3. As a result, mainstream social-science studies use that survey item to measure interpersonal trust in research on social capital and economic growth, contradicting narrower definitions that limit interpersonal trust to known individuals.

Psychology’s Greatest Misses (Part 1/3)

Living Fossils • 31 implied HN points • 07 Jan 26
🔬 Science Research Methods
  1. The replication crisis is mainly a failure of methods and weak evidence, not a need for new grand theories; psychology needs better procedures and rigor to make its findings reliable.
  2. Many popular psychology ideas are unsupported or oversimplified—common claims about reading instruction, power posing, stages of grief, stereotype threat, and transference often don’t hold up and can mislead practice.
  3. People’s responses depend more on the relationship and context than on fixed traits; concepts like attachment work better as changeable strategies that vary across situations.

Weight Loss Industry Big Think - Expanding the Market -Part 2

Weight and Healthcare • 459 implied HN points • 02 Dec 23
🏥 Health & Wellness Research Methods
  1. The weight loss industry strategically turns existing higher body weights into disease diagnoses to expand its market.
  2. They aim to exploit insurance coverage and exaggerate risks associated with higher weights to increase profits.
  3. The industry targets all age groups and sizes, pushing for wider definitions of overweight and obesity to sell interventions and expand its customer base.

Does birth order matter?

Wyclif's Dust • 2146 implied HN points • 11 Nov 23
🔬 Science Research Methods
  1. Birth order and parental age influence outcomes in opposite ways.
  2. Within families, birth order and parental age have a high correlation.
  3. Even though birth order effects are big, they explain very little of the variation in outcomes.

Should We Care About Replication?

Economic Forces • 6 implied HN points • 19 Feb 26
🔬 Science Research Methods
  1. Replication matters because it helps catch fraud and honest mistakes, but it doesn't have to be literal — redoing analyses with different data, larger samples, or better measures can serve the same purpose.
  2. A fall in published comments doesn't mean debate stopped; many critiques now happen in peer review and long appendices, and academic hierarchies and publication norms also shape what gets publicly challenged.
  3. Frontier empirical work is noisy and many surprising results won't generalize, so basic price theory and simple models are essential for asking better questions, judging results, and prioritizing what to replicate.

Dubious Justifications Behind Request to WHO to Declare Diet Drugs "Essential" Part 2

Weight and Healthcare • 479 implied HN points • 12 Apr 23
🏥 Health & Wellness Research Methods
  1. The request for WHO to declare diet drugs as essential is based on questionable justifications from studies with limited data and potential conflicts of interest.
  2. Global recommendations for diet drugs are primarily supported by research conducted on a predominantly white population, raising concerns about generalizability.
  3. Financial ties between pharmaceutical companies and organizations providing recommendations for diet drugs highlight potential biases and conflicts of interest in the decision-making process.

Spinning Bacteria

Asimov Press • 186 implied HN points • 31 Jul 25
🔬 Science Research Methods
  1. Scientists discovered how bacteriophages, or phages, time their escape from bacteria by spinning them. This helped them understand when phages decide to burst out after infecting a host.
  2. The research showed that phages wait until just the right moment, keeping the bacteria's energy intact until they are ready to reproduce as many phages as possible.
  3. Using a clever spinning technique and measuring the bacteria's electrical gradient, researchers found that if the charge drops by half, phages trigger lysis immediately, maximizing their chances of survival.

The Buckingham Manifesto for a Post-Progressive Social Science

Unsafe Science • 122 implied HN points • 04 Aug 25
🚌 Education Research Methods
  1. The social sciences have become too focused on progressive ideals, which can stifle diverse viewpoints and limit research. It's important to promote openness to different ideas and approaches in these fields.
  2. A fresh perspective called 'Critical Woke Studies' is needed to understand the rise of wokeness in academia and its impact on society. This study can help uncover the historical roots and motivations behind this ideology.
  3. There is a call for a new type of university or research center that values free inquiry and diverse opinions. This new approach could help rebuild trust in academic institutions and foster productive discussions.

New Vindication for the Regnerus Same-Sex Parenting Study

Unsafe Science • 103 implied HN points • 20 Aug 25
🇺🇸 U.S. Politics Research Methods
  1. The Regnerus study looked at the outcomes of children raised by same-sex parents compared to those with biological parents. It found that children with biological parents generally fared better across many areas.
  2. Recent analyses have confirmed the original findings of the Regnerus study, even under various scrutiny and different analytical methods. This suggests that the study's conclusions about family structures are more robust than previously thought.
  3. Critiques of the Regnerus study focused on its classifications and methodology, arguing they might have overstated the differences. However, newer analyses argue that both family transitions and the involvement of same-sex parents have measurable impacts on child outcomes.

"You Couldn't Replicate Our Study Because You're Ugly"

Cremieux Recueil • 416 implied HN points • 03 Dec 24
🔬 Science Research Methods
  1. Attractiveness studies may not be very reliable because their methods can be flawed. It's important to be careful about how these studies are designed and what they claim.
  2. Different studies use different ways to measure attractiveness, which can lead to confusion and mismatched results. It's not always clear which findings are valid.
  3. Racial preference in dating apps can be hard to measure correctly. Good research design is key, and many studies may not handle these issues well, leading to uncertain conclusions.

The Placebo Paradox: Why Placebos Are Ineffective In Psychedelic Research

Tripsitter • 79 implied HN points • 20 Mar 24
🏥 Health & Wellness Research Methods
  1. Researchers struggle with placebo controls in psychedelic trials due to the unmistakable subjective experiences and physiological effects of psychedelics, making blinding participants difficult.
  2. The power of expectations and beliefs influence the effectiveness of psychedelics, leading to challenges in maintaining placebo controls and interpreting trial results.
  3. Alternative approaches in psychedelic research involve creating placebo psychedelic experiences or comparing psychedelics directly to established treatments, offering potential solutions to traditional placebo limitations.

How I've been using ChatGPT

The Counterfactual • 219 implied HN points • 25 Jul 23
đź•ą Technology Research Methods
  1. ChatGPT can help you learn about new topics by suggesting useful resources and references. This can speed up your research by providing relevant information without the hassle of searching through many documents.
  2. Using ChatGPT for recommendations can be helpful, but it shouldn't replace getting suggestions from friends or experts. It can fill in gaps when you don't have access to personal recommendations.
  3. ChatGPT acts as a good reading companion by answering specific questions while you read. This helps you understand the material better and encourages you to ask questions about what you’re learning.

An Open Proposal to the CEO of the American Academy of Pediatrics

Steve Kirsch's newsletter • 7 implied HN points • 16 Jan 26
🏥 Health Politics Research Methods
  1. A plan to fund multiple independent research teams to analyze the same public government vaccine data, aiming to answer vaccine safety questions transparently.
  2. The effort would use jointly designed study specs, pre-registered protocols, and require public release of data, code, and methods, with publication of results regardless of outcome.
  3. The purpose is to reduce polarization and increase public trust by letting objective, independently verified analyses speak for themselves and involving organizations with different perspectives.

Eight questions "Professor Dave" can't answer

Steve Kirsch's newsletter • 7 implied HN points • 12 Jan 26
🏥 Health Politics Research Methods
  1. The debate became personal and some questions were deleted, raising concerns that ad hominem attacks and comment removal undermined open scientific discussion.
  2. There is a demand for clear, record-level evidence and transparent data (including autopsy results) to show whether COVID vaccines saved lives or caused harm, and a new analysis is claimed to show increased deaths after boosters.
  3. The speaker challenges the opponent's credibility and asks for credentials plus explanations for puzzling findings like odd protective effects in one study, sudden autism cases after vaccination, and unusual cancer trends.

Doesn't Being Higher Weight Create Higher Risk for Health Issues?

Weight and Healthcare • 559 implied HN points • 22 Jun 22
🏥 Health & Wellness Research Methods
  1. Correlation between weight and health doesn't imply causation, and it's crucial to examine the quality of evidence before drawing conclusions.
  2. Factors like weight stigma, weight cycling, and healthcare inequalities can influence the relationship between weight and health.
  3. Supporting health doesn't always need manipulation of body size, and it's essential to prioritize the rights of individuals regardless of their size.

Preregistration Is No Panacea

Cremieux Recueil • 283 implied HN points • 22 Nov 24
🔬 Science Research Methods
  1. Preregistration in research is not a guaranteed way to ensure the accuracy of results. Even when studies are preregistered, they can still produce results that don't hold up upon replication.
  2. Simply preregistering a study isn’t enough. There needs to be strict detail and oversight in how research is conducted to prevent 'p-hacking' or manipulating data for desired results.
  3. Focusing on predictive research rather than just explanatory studies can help ensure that research is more relevant and impactful. When studies predict future outcomes, they are more likely to contribute meaningfully to our understanding.

LatchBio Now Provides Lots of Data

LatchBio • 82 implied HN points • 27 Jun 25
🔬 Science Research Methods
  1. LatchBio has created a massive cell atlas with 30 million samples covering 150 diseases and 200 tissues. This helps researchers access diverse biological data easily.
  2. They partnered with Pythia Biosciences and Miraomics to enhance data curation and improve how this information is delivered to users.
  3. The introduction of a new Python framework helps scientists curate data more efficiently, making it easier to handle complex biological information.

Article Analysis: Should Type 2 Diabetes Management Focus on Weight Loss over Blood Sugar?

Weight and Healthcare • 419 implied HN points • 13 Jul 22
🏥 Health & Wellness Research Methods
  1. The article discusses the debate over whether Type 2 Diabetes management should prioritize weight loss or blood sugar control.
  2. It highlights the influence of pharmaceutical companies on medical advice and guidelines in the diabetes management space.
  3. The article raises concerns about financial conflicts of interest, lack of long-term efficacy data, and potential risks associated with focusing on weight loss over other health factors.

The difference between the US childhood vaccination schedule and Denmark's

Vinay Prasad's Observations and Thoughts • 195 implied HN points • 09 Dec 24
🏥 Health Politics Research Methods
  1. There are big differences between the childhood vaccination schedules in the US and Denmark. Studying these differences could help us understand which method works better for kids.
  2. A cluster randomized trial could test the effectiveness of the US vaccination schedule compared to Denmark's. This means grouping areas to follow one schedule or the other and then measuring health outcomes.
  3. There's a lot of uncertainty about which vaccination approach is best. By collecting better data, we can start to settle these important questions instead of just accepting existing beliefs.

The Counterfactual's year in review, and an announcement

The Counterfactual • 79 implied HN points • 29 Dec 23
🔬 Science Research Methods
  1. The Counterfactual had a successful year, growing its readership significantly after a popular post about large language models. It’s great to see how sharing knowledge can attract more people.
  2. Key posts focused on topics like construct validity and the understanding of large language models. These discussions are crucial for improving how we evaluate and understand AI technology.
  3. In 2024, the plan includes more posts and introducing paid subscriptions that allow subscribers to vote on future research projects. This will encourage community participation in exploring interesting ideas.

Understanding the Hawthorne Effect in UX Research

UX Psychology • 138 implied HN points • 06 Feb 23
đź•ą Technology Research Methods
  1. The Hawthorne Effect is when individuals change their behavior because they know they are being observed, impacting various behaviors from dietary habits to research study results.
  2. Possible explanations for the Hawthorne Effect include people conforming to expectations when observed and feeling pressured to perform better.
  3. To mitigate the Hawthorne Effect in UX research, steps like using control groups, minimizing feedback during studies, focusing on cause-and-effect relationships, and creating judgment-free environments can help obtain more accurate data.

Prediction Technologies and Innovation

New Things Under the Sun • 160 implied HN points • 08 Jan 25
đź•ą Technology Research Methods
  1. Prediction technologies help scientists make better guesses about what to explore next, like using AI to identify promising research areas. However, they can also lead people to focus too much on certain topics, missing out on other important areas.
  2. Research tools can change what scientists choose to study. For example, a tool might encourage research on proteins we already know about instead of new, less understood ones, which could slow down innovation.
  3. Different prediction technologies have different effects. Some can help researchers discover more unique solutions, while others may cause everyone to chase the same problems, limiting overall progress.

Why Beats What: Prioritize Qualitative User Research Over Quantitative

Jakob Nielsen on UX • 54 implied HN points • 31 Jul 25
đź•ą Technology Research Methods
  1. Focus on qualitative user research to understand why users succeed or fail with a design. This approach offers deeper insights than just looking at numbers.
  2. Testing with just a few users can uncover most usability problems. Don’t wait for large studies; small, quick tests can lead to improvements faster.
  3. Always watch what users actually do instead of relying on what they say. People often don't know why they struggle, so observing their behavior gives you better clues for design changes.

Metascience Manquè

Never Met a Science • 50 implied HN points • 17 Jul 25
🔬 Science Research Methods
  1. Metascience is struggling because it's too focused on just replicating studies, rather than exploring deeper questions about what science is and how it could improve. This limits its potential.
  2. Communication between different scientific fields can be helpful, but it often leads to misunderstandings about methods and goals. This can result in a distorted view of what science really is.
  3. To make science better, we need to rethink how we share knowledge. Relying only on traditional formats like PDFs isn't working anymore. More flexible and collaborative ways of sharing research could lead to better science.

The peer-review literature says science should not run from public challenges; here's a framework for fair discussions

Steve Kirsch's newsletter • 5 implied HN points • 01 Jan 26
🔬 Science Research Methods
  1. Experts should engage in public, evidence-based debate instead of avoiding challenges. Refusing to debate undermines scientific norms and public trust.
  2. A fair public debate needs a narrow question, balanced credentialed panels with conflict disclosures, a strict evidence-first format, and an independent post-debate audit, with all data and the full record publicly archived. Neutral funding and platform transparency are also required.
  3. Relying only on closed peer review and avoiding open scrutiny weakens credibility. Reforming peer review and embracing transparent, verifiable debates would better hold researchers accountable and inform the public.

Cognitive Enhancement as Research Priority

Parrhesia • 608 implied HN points • 12 May 23
🔬 Science Research Methods
  1. Consider the most critical areas of inquiry for resource-efficient research
  2. Meta-discoveries like artificial general intelligence could be the pinnacle of scientific progress
  3. Be cautious about accelerating machine intelligence and prioritize human cognitive enhancement for intellectual advancement

~75% of Psychology Claims are False

Unsafe Science • 161 implied HN points • 01 Nov 24
🔬 Science Research Methods
  1. A lot of psychology studies are not reliable, with about 75% of their claims likely being false. So, it's best to be skeptical of what you read in this field.
  2. Many published psychology findings are difficult to reproduce. This means that even though a study shows something, there's a good chance it might not be true when tried again.
  3. Censorship and ignoring opposing evidence can lead to more misinformation in psychology. Some studies get more attention even when their findings are wrong or contested.