Darpa Wants to Solve Science's Replication Crisis With Robots

WIRED | 2/15/2019 | Adam Rogers
roxy2707 (Posted by) Level 3
Click For Photo: https://media.wired.com/photos/5c6604ef84b7ee0c65179b87/191:100/pass/Validity-and-Reproducibility_Final.jpg

Say this much for the “reproducibility crisis” in science: It’s poorly timed. At the same instant that a significant chunk of elected and appointed policymakers seem to disbelieve the science behind global warming, and a significant chunk of parents seem to disbelieve the science behind vaccines … a bunch of actual scientists come along and point out that vast swaths of the social sciences don’t stand up to scrutiny. They don’t replicate—which is to say, if someone else does the same experiment, they get different (often contradictory) results. The scientific term for that is bad.

What’s good, though, is that the scientific method is built for self-correction. Researchers are trying to fix the problem. They’re encouraging more sharing of data sets and urging each other to preregister their hypotheses—declaring what they intend to find and how they intend to find it. The idea is to cut down on the statistical shenanigans and memory-holing of negative results that got the field into this mess. No more collecting a giant blob of data and then combing through it for a publishable outcome, a practice known as “HARKing”—hypothesizing after results are known.

Teams - Work - Experiment - Effect - Slog—boring

And self-appointed teams are even going back through old work, manually, to see what holds up and what doesn’t. That means doing the same experiment again, or trying to expand it to see if the effect generalizes. It’s a slog—boring, expensive, and time-consuming. To the Defense Advanced Research Projects Agency, the Pentagon’s mad-science wing, the problem demands an obvious solution: Robots.

“The replication work is an assessment of ground-truth fact,” a final call on whether a study held up or failed, says Tim Errington, director of research at COS. “That’s going to get benchmarked against algorithms. Other teams are going to come up with a way to do that automatically, and then you assess...
(Excerpt) Read more at: WIRED
Wake Up To Breaking News!
Sign In or Register to comment.

Welcome to Long Room!

Where The World Finds Its News!