200 Researchers, 5 Hypotheses, No Consistent Answers

Wired | 12/6/2019 | Staff
Click For Photo: https://media.wired.com/photos/5de99233b4ac1c00080cd539/191:100/w_1280,c_limit/Ideas_ScientistsHypothesis_Template.jpg

If science is an objective means of seeking truth, it’s also one that requires human judgments. Let’s say you’re a psychologist with a hypothesis: People understand that they may be biased in unconscious ways against stigmatized groups; they will admit this if you ask them. That might seem like a pretty straightforward idea—one that’s either true or not. But the best way to test it isn’t necessarily obvious. First, what do you mean by negative stereotypes? Which stigmatized groups are you talking about? How would you measure the extent to which people are aware of their implicit attitudes, and how would you gauge their willingness to disclose them?

These questions could be answered in many different ways; these, in turn, may lead to vastly different findings. A new crowdsourced experiment—involving more than 15,000 subjects and 200 researchers in more than two dozen countries—proves that point. When various research teams designed their own means of testing the very same set of research questions, they came up with divergent, and in some cases opposing, results.

Study - Demonstration - Idea - Light - Reproducibility

The crowdsourced study is a dramatic demonstration of an idea that’s been widely discussed in light of the reproducibility crisis—the notion that subjective decisions researchers make while designing their studies can have an enormous impact on their observed results. Whether through p-hacking or via the choices they make as they wander the garden of forking paths, researchers may intentionally or inadvertently nudge their results toward a particular conclusion.

The new paper’s senior author, psychologist Eric Uhlmann at INSEAD in Singapore, had previously spearheaded a study that gave a single data set to 29 research teams and asked them to use it to answer a simple research question: “Do soccer referees give more red cards to dark-skinned players than light-skinned ones?” Despite analyzing identical data, none of the teams came up...
(Excerpt) Read more at: Wired
Wake Up To Breaking News!
Hell sometimes looks an awful lot like an office cubicle.
Sign In or Register to comment.

Welcome to Long Room!

Where The World Finds Its News!