A replication of a 2023 neuroscience study on brain entrainment and learning found that the original effect likely doesn't exist, revealing how the study obscured key issues through statistical averaging and highlighting how AI tools are democratizing scientific scrutiny. Longer summary

Sasha Putilin received an ACX grant to replicate a 2023 study claiming that flickering lights synchronized to individual brain rhythms could boost learning speed threefold. The $32,000 replication with 12 participants (versus 80 in the original) found no such effect. Upon examining the original data more closely, Putilin discovered the apparent effect was driven entirely by a few participants with large negative learning rates (getting worse over time) who happened to all be in one experimental group. The original study had obscured this pattern by presenting averaged data rather than individual results. Putilin argues this exemplifies 'cargo-cult statistics' where researchers mechanically apply statistical rituals without critically examining underlying data. The post concludes by suggesting AI tools are democratizing meta-science, enabling anyone to audit published research that previously required expert-level effort. Shorter summary

Scott Alexander proposes various project ideas for ACX Grants, ranging from scientific research to social initiatives, and encourages readers to pursue them. Longer summary

Scott Alexander proposes several project ideas for the upcoming ACX Grants round. These range from replicating brain entrainment learning results to creating an open-source polygenic score for educational attainment. Other suggestions include campaigns similar to John Green's anti-tuberculosis effort, a novel approach to language teaching, an automatic Implicit Association Test generator, a new dating site concept, a foundation to promote classical art and architecture, and a primer on political change. Scott encourages readers to take on these projects, offering potential publicity and connections to funders. Shorter summary