Skip to main content
All Reviews
PsychologyNiche
beginner

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

Richard A. Klein et al. (2018)

Published
Dec 1, 2018
Journal
Advances in Methods and Practices in Psychological Science · Vol. 1 · No. 4
DOI
10.1177/2515245918810225

Summary

ML2 replicated 28 effects across 125 samples spanning 36 countries. 14 of 28 (50%) replicated by the traditional significance criterion. The key finding: variation in effect sizes was mostly due to which effect was being tested, not where or by whom. The "hidden moderators" defense (failures to replicate are due to unidentified contextual differences between samples) received little empirical support. When effects failed, they failed consistently across diverse settings.

Method Snapshot

Crowdsourced multi-lab direct replication

Background

Their first manuscript is essential to read before

In all our papers, we write that you can't generalize the conclusions to the population at large. However, apparently, that's not the problem: if the effect exists, then it works everywhere!

ES