A replication of a 2023 neuroscience study on brain entrainment and learning found that the original effect likely doesn't exist, revealing how the study obscured key issues through statistical averaging and highlighting how AI tools are democratizing scientific scrutiny. Longer summary

Sasha Putilin received an ACX grant to replicate a 2023 study claiming that flickering lights synchronized to individual brain rhythms could boost learning speed threefold. The $32,000 replication with 12 participants (versus 80 in the original) found no such effect. Upon examining the original data more closely, Putilin discovered the apparent effect was driven entirely by a few participants with large negative learning rates (getting worse over time) who happened to all be in one experimental group. The original study had obscured this pattern by presenting averaged data rather than individual results. Putilin argues this exemplifies 'cargo-cult statistics' where researchers mechanically apply statistical rituals without critically examining underlying data. The post concludes by suggesting AI tools are democratizing meta-science, enabling anyone to audit published research that previously required expert-level effort. Shorter summary

Scott Alexander's experiment tested how different essays affect people's concerns about AI risk, finding a modest but persistent increase in concern after reading. Longer summary

Scott Alexander conducted an experiment to test the effectiveness of different essays in persuading people about AI risk. Participants were assigned one of five essays to read, including a control essay unrelated to AI. The main outcome was participants' level of concern about AI risk on a 1-10 scale. Results showed that reading the AI-related essays increased concern by an average of 0.5 points, with no significant differences between the four AI essays. The effect persisted at about two-thirds strength after one month. The experiment also looked at secondary outcomes related to specific AI risk questions and analyzed differences based on prior familiarity with the topic. Overall, the study suggests a modest but useful effect from trying to persuade people through essays on this topic. Shorter summary

Scott Alexander analyzes two conflicting studies on gender bias in STEM hiring, exploring reasons for their contradictory results and the challenges in reaching a definitive conclusion. Longer summary

Scott Alexander discusses two contradictory studies on gender bias in STEM hiring, one showing bias against women and another showing bias in favor of women. He explores possible reasons for the discrepancy, including differences in methodology and potential experimenter bias. The post highlights the difficulty in reaching a definitive conclusion on this issue despite its importance and the resources dedicated to studying it. Scott suggests that the conflicting results might be due to subtle experimenter effects and proposes a joint study by both teams as a potential solution. Shorter summary

A person repeatedly wakes up in a Matrix-like pod, being told different versions of reality about the concept of 'family' in a series of nested scenarios, ultimately revealed to be an alien experiment. Longer summary

This fictional story describes a series of nested simulations or scenarios where the protagonist repeatedly wakes up in a Matrix-like pod. Each time, they are told a different version of reality regarding the concept of 'family'. The story explores themes of conformity, belief systems, and the malleability of memory and perception. It ends with a twist revealing that the entire experiment was conducted by aliens seeking to understand optimal social arrangements, particularly regarding the concept of family. Shorter summary

Scott attempts to test a productivity hack using calligraphy study, but discovers calligraphy itself is captivating, leaving the experiment inconclusive. Longer summary

Scott Alexander humorously recounts his attempt to test a productivity hack starting with 'M' using calligraphy study as a metric. He ends up spending seven hours studying calligraphy uninterrupted during the day, and another five hours late at night. Scott concludes that calligraphy is inherently fascinating, rendering his experiment inconclusive about the effectiveness of the productivity technique. Shorter summary