Want to dive into Scott Alexander's work and his thousands of blog posts? This fan website lets you sort and do semantic search through the whole codex. Enjoy!

See also Top Posts and All Tags.

Tag: publication bias

Minutes:
Pick a custom range (minutes). Leave a field empty for no limit.
Blog:
Year:
2026
2025
2024
2023
2022
2021
2020
2019
2018
2017
2016
2015
2014
2013
Tags:
Filter by tag...
Exclude tag...
5212 tags
Links:
Filter by linked site (twitter, substack…)
7 posts found
Compact Mode
Save Reads
Feb 01, 2023
acx
Read on
86 min 13,307 words 315 comments 105 likes podcast (79 min)
Scott Alexander responds to critiques of his 2021 ivermectin analysis, acknowledging some errors but maintaining his conclusion that ivermectin likely doesn't work for COVID-19. Longer summary
Scott Alexander responds to Alexandros Marinos' critique of his 2021 post on ivermectin studies, addressing points about individual studies, meta-analysis methods, publication bias, and new evidence since 2021. He acknowledges some mistakes in his original analysis but maintains his overall conclusion that ivermectin is likely ineffective for COVID-19. Shorter summary
Sep 05, 2015
ssc
Read on
13 min 1,866 words 318 comments
Scott Alexander argues that psychology is indeed in crisis, contrary to a New York Times article's claim, due to issues like publication bias and low replication rates. Longer summary
Scott Alexander critiques a New York Times article claiming psychology is not in crisis despite low replication rates. He argues that the article ignores publication bias, experimenter effects, and low base rates of true hypotheses. Scott contends that even if failed replications are due to different conditions, this still represents a crisis as it undermines the practical utility of psychological findings. He suggests that while we can't investigate every failed replication, studying some might reveal why replication issues keep occurring in psychology. Shorter summary
Apr 22, 2015
ssc
Read on
12 min 1,709 words 254 comments
The post deconstructs a growth mindset study, revealing that its positive claims in the abstract are not supported by the actual data, demonstrating the importance of critical analysis of scientific papers. Longer summary
The post critically analyzes a study on growth mindset interventions, revealing that the abstract's positive claims are not supported by the actual data. The author demonstrates how the study's results, when examined closely, show little to no effect of growth mindset interventions on students' academic performance. The post highlights the importance of reading beyond abstracts and titles, and critically examining scientific papers, especially in popular fields where publication bias may exist. The author uses this example to illustrate how careful analysis can reveal contradictions between a study's stated conclusions and its actual findings. Shorter summary
Scott Alexander examines a study comparing the effectiveness of drugs and therapy for psychiatric disorders, discussing the results and methodological limitations of the research. Longer summary
This post analyzes a study comparing the efficacy of pharmacotherapy and psychotherapy for various psychiatric disorders. The author discusses the graph showing effect sizes for different treatments, noting that most psychiatric treatments have an effect size around 0.5. He expresses some uncertainty about the statistical methods used and highlights three surprising findings: drugs appearing more effective than therapy for borderline personality disorder and insomnia, and drugs being more effective at preventing relapse than stopping acute episodes. The post also discusses the limitations of psychotherapy trials, noting that lower quality trials tend to show much higher effect sizes than high-quality ones, and that psychotherapy research often lacks sufficient blinding and control groups. Shorter summary
Dec 12, 2014
ssc
Read on
19 min 2,880 words 270 comments podcast (22 min)
Scott Alexander cautions against basing opinions on limited research, using examples from medicine and economics to show how cherry-picking studies can lead to opposing conclusions. Longer summary
Scott Alexander warns against relying on a single study or a limited selection of studies to form opinions on complex issues. He illustrates this with examples from medical research and the minimum wage debate, showing how cherry-picking studies can lead to opposing conclusions. The post emphasizes the importance of considering the full body of evidence, including meta-analyses and expert opinions, while also being aware of potential biases in research and reporting. Scott concludes by advocating for skepticism and thorough investigation when evaluating claims backed by scientific studies. Shorter summary
Jul 07, 2014
ssc
Read on
56 min 8,645 words 129 comments
Scott analyzes various criticisms of antidepressants, concluding they have modest but real benefits over placebo, with important considerations about side effects and efficacy. Longer summary
This post examines various criticisms of SSRIs and antidepressants, addressing claims about their efficacy, side effects, and comparisons to placebo and psychotherapy. Scott analyzes studies on antidepressant effectiveness, discussing issues like publication bias, effect sizes, and the meaning of 'clinical significance'. He explores side effects, particularly sexual dysfunction, weight gain, and emotional blunting. The post concludes that while antidepressants are not miracle drugs, they do have a modest but statistically significant benefit over placebo and can be a reasonable option for many people with depression, especially if they understand and prepare for potential side effects. Shorter summary
Feb 17, 2013
ssc
Read on
24 min 3,699 words 26 comments
Scott Alexander examines the claim that '90% of medical research is false', arguing it's an exaggeration while acknowledging real issues in the field. Longer summary
Scott Alexander critiques the popular claim that '90% of medical research is false', which is often attributed to John Ioannidis. He argues that this statement, while pointing to important issues, creates more panic than warranted. Scott analyzes Ioannidis' work, showing that the 90% figure is likely misinterpreted from various sources. He explains that the accuracy of medical research varies greatly depending on the type of study, with large randomized trials and meta-analyses being much more reliable. Scott also discusses how multiple studies on the same topic can greatly increase confidence in results, and how doctors' beliefs are typically based on substantial evidence rather than single studies. He concludes by acknowledging the problems in medical research while cautioning against overly cynical interpretations. Shorter summary
Per page:
Showing 1 to 7 of 7 results
Get these search results in an EPUB

Your filters match 7 posts.

Posts to include
Leave empty to keep the defaults. Range cannot exceed 500 posts.
Download now

Generates an EPUB right now and downloads it to your device.

Send to email

Generates an EPUB in the background and emails you a temporary download link.

Your email is not shared with anyone.

Email address

To send to your Kindle, just use this link.