Scott Alexander reviews two papers exposing statistical manipulation techniques in psychology research and addiction treatment program evaluations.
Longer summary
This post discusses two papers on statistical manipulation in scientific studies. The first paper, 'False Positive Psychology', demonstrates how researchers can use four tricks to artificially achieve statistical significance: measuring multiple dependent variables, choosing when to end experiments, controlling for confounders, and testing different conditions. The authors show these tricks can make random data appear significant 61% of the time. The second paper, 'How To Have A High Success Rate In Treatment', reveals how addiction treatment programs can inflate their success rates through various methods like carefully choosing the denominator, selecting promising candidates, redefining success, and omitting control groups. Both papers highlight the ease of manipulating statistics to produce desired results in research and treatment evaluations.
Shorter summary