Fix Science In Half An Hour

You’ve probably heard about the crisis of replication in psychology. The problem is that replication is an unglamorous business; researchers would much rather do the sexier work of pushing forward knowledge with new results.

So we need to make replications more glamorous.

I propose a reality TV show, Replication Lab!, where every week they try to replicate one of the most famous experiments from the past few years.

It starts with the host explaining the experiment, maybe an interview with a very distinguished elderly professor who talks about how confident he is that his results will hold up. The techs chat with each other as they construct the experimental setup about how they’re doing and how their date last night went and how they’re going to avoid the problems that confounded the original study.

Suspense builds as we see the participants come in. Some human interest stories. He agreed to participate because they offered $30, which he’s going to use to buy a present that will win back his estranged daughter’s love. She joined because she’s right on the border of failing her psych class and needs the extra credit to save her dream of becoming the first person in her family to graduate college.

The experiment itself. The suspense is unbearable. We get a running commentary as everything proceeds. Oh man, look how harsh that guy is being on his Milgram Obedience Experiment, can you believe he would do that? That girl in the control condition seems to be running through her Stroop task at lightning speed – how do you think that’s going to affect our results, kindly-looking bearded scientist attached to the show?

After a tension-building commercial break, we get the results. Everyone is huddled around a computer as the statistician makes the final mouse click, and…oh no, p = .30! Total failure to replicate!

The scene cuts to the distinguished elderly professor’s face as he sees his great discovery going down the toilet. “How do you feel right now?” asks the host, and the professor sputters “I…I’m sure time will vindicate me! I know it!” and then he runs off the set, crying. Our host turns to the kindly-looking bearded scientist attached to the show. “Tell me the truth,” she says “Do you think Dr. Zuckerman’s career is ruined?” “I can’t imagine it wouldn’t be,” says the bearded scientist, shaking his head sadly.

I feel like Mythbusters has probably pretty much exhausted our cultural stock of urban legends by now and could be profitably recruited for this project. I would also accept “Welcome to Replication Lab! With your host, John Ioannidis!”