A replication of a 2023 neuroscience study on brain entrainment and learning found that the original effect likely doesn't exist, revealing how the study obscured key issues through statistical averaging and highlighting how AI tools are democratizing scientific scrutiny. Longer summary

Sasha Putilin received an ACX grant to replicate a 2023 study claiming that flickering lights synchronized to individual brain rhythms could boost learning speed threefold. The $32,000 replication with 12 participants (versus 80 in the original) found no such effect. Upon examining the original data more closely, Putilin discovered the apparent effect was driven entirely by a few participants with large negative learning rates (getting worse over time) who happened to all be in one experimental group. The original study had obscured this pattern by presenting averaged data rather than individual results. Putilin argues this exemplifies 'cargo-cult statistics' where researchers mechanically apply statistical rituals without critically examining underlying data. The post concludes by suggesting AI tools are democratizing meta-science, enabling anyone to audit published research that previously required expert-level effort. Shorter summary

Scott examines whether reported increases in "crime" actually reflect concerns about "disorder" (litter, graffiti, shoplifting, homelessness, tent encampments), finding that most disorder metrics are either stable or down from historical highs, contradicting narratives of societal collapse. Longer summary

Scott investigates the theory that public concern about rising crime is actually proxy concern about rising disorder (litter, graffiti, shoplifting, homelessness, tent encampments, boom boxes). He systematically examines data for each type of disorder and finds that most are either down from historical highs or only modestly increased from recent lows. Litter is down since the 1970s, graffiti is unclear but likely down in most places, shoplifting is up 20% from lows but still below 1990s levels, and homelessness is up 25% but equal to 1990s levels. He proposes four theories for why people perceive disorder as rising: the small 2020 bump, white people returning to cities through gentrification, romanticization of the unique 1930s-1960s crime minimum, and different forms of disorder in past versus present. Scott concludes by distinguishing between specific, evidence-based concerns about particular problems versus vague civilizational collapse narratives, arguing the former is both more accurate and more conducive to normal life. Shorter summary

Scott Alexander summarizes comments on his healthcare effectiveness debate with Robin Hanson, including clarifications, study author responses, and reader perspectives on various healthcare topics. Longer summary

This post highlights comments on Scott Alexander's recent discussion with Robin Hanson about healthcare effectiveness. It includes responses from Robin Hanson clarifying his position, comments from the authors of a study on health insurance and mortality, and various reader perspectives on healthcare, medical waste, and specific medical treatments. The post touches on topics like the statistical analysis of healthcare studies, the value of preventive medicine, and the complexities of measuring healthcare outcomes. Shorter summary

Scott Alexander explores correlations between PMS symptoms and various beliefs and traits, finding significant connections that leave him puzzled. Longer summary

Scott Alexander discusses survey results related to PMS symptoms and their correlations with various beliefs and traits. He attempts to replicate findings from Aella's Twitter polls, which showed connections between PMS and belief in the supernatural, as well as neuroticism. Using data from the ACX survey, Scott finds significant correlations between PMS symptoms and belief in the supernatural, as well as anxiety levels. He explores potential explanations, including cultural influences and hormonal factors, and presents additional correlations found in the survey data. The post ends with Scott expressing confusion about the findings and inviting readers to replicate his work using the publicly available survey results. Shorter summary

Scott Alexander challenges Robin Hanson's claim that medicine doesn't work by analyzing health insurance studies and presenting evidence of medicine's effectiveness. Longer summary

Scott Alexander critiques Robin Hanson's claim that medicine doesn't work, analyzing three major health insurance experiments (RAND, Oregon, and Karnataka) and other studies. He argues that these studies are underpowered to detect medication effects and don't support Hanson's conclusion, citing evidence of medicine's effectiveness in improving survival rates for various diseases. Shorter summary

Scott Alexander investigates why self-reported IQ scores in online surveys are implausibly high, identifying issues with conversion tools, reporting bias, and unreliable tests. Longer summary

Scott Alexander investigates why self-reported IQ scores in online surveys seem implausibly high. He analyzes data from two surveys (Less Wrong 2014 and Clearer Thinking 2023) and identifies three main issues: 1) A popular SAT to IQ conversion tool was inaccurate, inflating scores. 2) Only the smartest people tend to report their SAT scores, skewing the sample. 3) Self-reported IQ test scores are often inflated or based on unreliable tests. After accounting for these factors, he estimates the average IQ of the Clearer Thinking sample at 111 and the Less Wrong sample at 128, which he considers more plausible. Shorter summary

A critical review of 'Why Nations Fail' that examines the authors' academic work and methodology, while questioning the book's quality and practical implications. Longer summary

This review critiques 'Why Nations Fail' by Acemoglu and Robinson (AR), arguing that while the book itself is not very good, AR's academic papers present a more compelling case for their thesis that political institutions are the main determinant of economic growth. The reviewer explains AR's methodology using instrumental variables, discusses their evidence and criticisms, and ultimately concludes that even if AR are correct, their extremely long-run focus may not be very useful for practical policymaking or development efforts. Shorter summary

The post challenges the notion that effect sizes below 0.50 for medications are clinically insignificant, using antidepressants as a case study and comparing them to other common drugs. Longer summary

This post discusses the interpretation of effect sizes in medical studies, particularly for antidepressants. It challenges the common notion that effect sizes below 0.50 are clinically insignificant. The author presents a Danish study that simulates various hypothetical medications, showing that even drugs that cure or significantly improve depression in a large percentage of patients might not meet traditional effect size thresholds. The post explains why these thresholds might be unrealistic due to factors like the placebo effect, high standard deviations in placebo groups, and patient dropouts. It also compares antidepressant effect sizes to those of other widely accepted medications, concluding that many common drugs would be considered 'clinically insignificant' by strict effect size standards. The author suggests that claims about 'meaningless effect sizes' should be given less weight compared to other evidence, such as clinical experience. Shorter summary

Scott fact-checks and disproves a claim that all healthy people have mystical experiences using SSC Survey data. Longer summary

Scott Alexander fact-checks a tweet claiming all healthy people have mystical experiences using data from the 2020 SSC Survey. He defines a 'very mentally healthy' category and compares their reported spiritual experiences to the general survey population. The analysis shows that very mentally healthy individuals are actually less likely to report spiritual experiences, disproving the tweet's claim. Scott expresses relief at this finding, noting that spiritual experiences can be helpful for people during difficult times. Shorter summary

Scott Alexander investigates the reliability of dietary supplements, finding that while issues exist, most are not completely fake as some media reports claim. Longer summary

Scott Alexander examines the trustworthiness of dietary supplements, debunking widespread claims that most are completely fake. He analyzes studies, consumer lab reports, and industry insider accounts to conclude that while there are issues in the supplement industry, outright fraud is rare, especially for common vitamins and minerals. More complex botanicals and trendy supplements may have more variability, but complete absence of active ingredients is uncommon. Scott suggests consumers should be discerning, especially with lesser-known products, but notes that for many supplements, precise dosing may not be critical. Shorter summary

Scott Alexander examines the poor quality of research on homework effectiveness, finding only one well-designed study showing positive effects for high school algebra. Longer summary

Scott Alexander discusses the lack of reliable research on the effectiveness of homework. He critiques existing studies for their flawed methodologies, particularly their reliance on self-reported time spent on homework as a proxy for homework amount. The post highlights issues with confounding factors and poor study designs. Alexander finds only one well-designed, randomized study on homework effectiveness, which shows a positive effect for 9th-grade algebra homework. However, he notes that this single study doesn't provide enough evidence to draw broad conclusions about homework effectiveness across different subjects and grade levels. Shorter summary

Scott Alexander shares 34 diverse and interesting links on topics ranging from historical curiosities to current scientific debates, with brief commentaries on each. Longer summary

This post is a collection of 34 interesting links and brief commentaries on various topics. It covers a wide range of subjects including historical anecdotes, scientific studies, economic theories, technological developments, and social issues. The author, Scott Alexander, provides his thoughts and sometimes skeptical analysis on many of the items. The links are diverse, ranging from a list of games Buddha wouldn't play to discussions about AI safety research opportunities. Shorter summary

Scott Alexander responds to comments on his ivermectin article, discussing criticisms, the worms hypothesis, vaccine skepticism, and journalism quality. Longer summary

This post highlights comments on Scott Alexander's previous article about ivermectin as a potential COVID-19 treatment. Scott responds to criticisms from ivmmeta.com, discusses the worms hypothesis with various commenters, addresses concerns about the TOGETHER trial, and shares perspectives on vaccine skepticism and journalism quality. He also reflects on his own writing process compared to traditional journalism. Shorter summary

Scott summarizes reader comments on his 'Secrets of the Great Families' post, covering additional examples, statistical discussions, and personal perspectives on family achievement. Longer summary

This post highlights comments on Scott's previous article about famous families. It includes additional examples of notable families, discussions on the statistical significance of family clustering, insights from people with high-achieving families, perspectives from those with less successful backgrounds, and various other related topics. The post covers a wide range of fields including science, sports, literature, and politics, and discusses factors like genetics, environment, and family culture that may contribute to success across generations. Shorter summary

Scott examines data on COVID-19 lockdown effectiveness, finding they likely reduced transmission but at significant costs, with targeted measures being most effective. Longer summary

Scott analyzes the effectiveness of COVID-19 lockdowns, examining data from Sweden, US states, and Europe. He finds that lockdowns likely reduced transmission rates, with targeted measures like school closures being more effective than stay-at-home orders. The analysis suggests lockdowns saved lives but at significant economic and emotional costs, with the cost-effectiveness varying between countries and time periods. Scott concludes that fast, well-targeted lockdowns may have been optimal, but emphasizes the high uncertainty in these estimates. Shorter summary

Scott Alexander directs readers to his essay on Works in Progress examining why suicide rates didn't increase during the COVID-19 pandemic as expected. Longer summary

Scott Alexander refers readers to his essay published on Works in Progress, which discusses the unexpected trend of suicides during the COVID-19 pandemic. The essay, titled 'Why didn't suicides rise during Covid?', likely explores the counterintuitive finding that suicide rates did not increase as many had predicted during the global health crisis. While the post itself doesn't provide details, it suggests an in-depth analysis of suicide statistics, mental health trends, and potential explanations for this phenomenon during the pandemic. Shorter summary

Scott Alexander analyzes a study suggesting cloth masks may be harmful, but remains cautiously in favor of their use despite the uncertainty. Longer summary

Scott Alexander examines a 2015 study by MacIntyre et al. on the efficacy of cloth masks compared to surgical masks and no masks in healthcare settings. The study found cloth masks performed worse than both surgical masks and the control group, raising concerns about their safety. However, Scott notes several limitations of the study and its interpretation. He leans towards believing cloth masks are still better than nothing, especially for blocking outgoing pathogens, but emphasizes the high uncertainty in this life-or-death decision during the COVID-19 pandemic. Shorter summary

Scott Alexander reviews John Gottman's marriage counseling methods, finding them intuitively appealing but scientifically questionable. Longer summary

Scott Alexander reviews John Gottman's book 'The Seven Principles For Making Marriage Work', summarizing Gottman's methods and then critically examining the scientific evidence behind them. The review begins with an overview of Gottman's reputation and claims, then details his approach to marital counseling, which focuses on liking your spouse, communication skills, and dealing with conflicts. Scott then investigates the statistical validity of Gottman's methods, finding significant issues with his claims of predictive accuracy and effectiveness. The review concludes with Scott's personal reflections on the challenges of relationship counseling and the nature of marital problems. Shorter summary

The post critically examines the impact of India's 1991 economic reforms, suggesting that the country's economic growth began earlier and that the causes are more complex than commonly believed. Longer summary

This post examines the economic reforms in India and their impact on poverty reduction and economic growth. The author challenges the widely held belief that the 1991 reforms were the primary cause of India's economic boom. By analyzing various studies and economic data, the post suggests that India's growth trajectory began earlier, possibly in the mid-1970s or early 1980s. The exact causes of this growth remain unclear, with various factors such as trade liberalization, public investment, and political attitudes towards business being potential contributors. The author also discusses the potential impact of socialist policies on developing countries and the importance of understanding these economic shifts for future policy-making and arguments against certain forms of socialism. Shorter summary

Scott Alexander attempts to replicate a study linking beef jerky consumption to manic episodes in bipolar disorder, finding no significant relationship in his analysis of SSC survey data. Longer summary

Scott Alexander attempts to replicate a study that claimed beef jerky consumption could trigger manic episodes in bipolar disorder. He uses data from the 2019 Slate Star Codex survey to compare beef jerky consumption between people with and without a history of bipolar mania hospitalization. Contrary to the original study, Scott's analysis finds no significant relationship between beef jerky consumption and manic episodes. He discusses the limitations of his data, such as relying on self-reporting and having a smaller sample size of manic patients, but also notes some strengths like a more detailed measure of beef jerky consumption. Scott concludes by urging further research on this topic, highlighting the importance of replication in scientific studies. Shorter summary

Scott Alexander critiques a misreported study on trigger warnings, highlighting its flaws and limited relevance to the broader debate on their use in colleges. Longer summary

Scott Alexander critiques a study on trigger warnings that has been misreported in media. He points out several flaws in the study, including weak statistical significance, inappropriate sample demographics, and questionable measurement of harm. Scott emphasizes that the study doesn't actually measure long-term effects or real-world impacts of trigger warnings on college students. He argues that the study's findings are being overinterpreted and misused in public discourse. The post ends with Scott reiterating his own proposal for implementing trigger warnings in a low-key, matter-of-fact manner. Shorter summary

Scott Alexander examines conflicting narratives about cancer treatment progress, analyzing data to show that while some improvements are due to social factors, there's evidence of genuine advancements in cancer treatment and outcomes. Longer summary

This post examines the progress made in the 'War on Cancer' by analyzing cancer incidence rates, mortality rates, and five-year survival rates. Scott Alexander explores two competing narratives: one suggesting significant progress and another claiming limited success. He delves into the data, considering factors like age adjustment, smoking trends, and early detection biases. The analysis reveals that while some improvements are due to social factors like reduced smoking, there's evidence of genuine progress in cancer treatment. Stage-specific survival rates have improved, and studies adjusting for various confounders still show advancements. The post concludes that while progress may not meet the grand expectations set in the past, there has been a gradual but real improvement in cancer treatment and outcomes. Shorter summary

Scott Alexander explores why US suicide rates aren't higher despite high gun ownership, concluding that the US likely has a low base rate elevated by gun availability. Longer summary

Scott Alexander investigates why the US doesn't have higher suicide rates despite high gun ownership. He examines data showing guns increase suicides within the US, but the US doesn't have higher rates internationally. Scott proposes that the US may have a very low base suicide rate due to factors like religiosity and ethnic diversity, which is then elevated by gun availability. He concludes that a gun-free US might have one of the lowest suicide rates in the developed world, around 5-6 per 100,000. The post warns against simplistic 'gotcha' arguments using international comparisons without deeper analysis. Shorter summary

Scott Alexander critically examines a major meta-analysis on antidepressant efficacy, noting potential biases and comparing its surprising drug rankings to his own previous analysis. Longer summary

This post reviews a major meta-analysis by Cipriani et al on the efficacy of antidepressants. The study claims to definitively show antidepressants work, but Scott notes it doesn't actually refute previous critiques about their effectiveness. He examines potential biases and methodological issues in the study, particularly around industry funding of trials. Scott also discusses the study's ranking of different antidepressants, noting some matches with conventional wisdom but also some surprising results. He compares these rankings to his own previous analysis, finding major discrepancies, and concludes by urging some caution in interpreting the study's results despite its impressive scope. Shorter summary

Scott Alexander examines the rise, fall, and alleged resurrection of the antidepressant NSI-189, expressing skepticism about its efficacy and the broader challenges in developing new antidepressants. Longer summary

Scott Alexander discusses the disappointing results of NSI-189, a promising new antidepressant, in FDA trials. He explains how the drug initially failed to outperform placebo on the primary endpoint, leading to a stock crash for Neuralstem. However, the company later released positive secondary endpoint results, causing their stock to rise again. Scott expresses skepticism about these new results, suggesting they may be due to statistical manipulation rather than true efficacy. He reflects on the challenges of developing new antidepressants and the tendency to get overly excited about new drugs, ending with a sardonic hope for another new antidepressant, SAGE-217. Shorter summary