Skip to main content
All Reviews
PsychologyWorth Reading
beginner

Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

Colin F. Camerer et al. (2018)

Published
Aug 27, 2018
Journal
Nature Human Behaviour · Vol. 2 · No. 9
DOI
10.1038/s41562-018-0399-z

At a Glance

Even Nature and Science papers replicate only 62% of the time — and at half the original effect size.

SummaryAI

The Social Science Replication Project replicated 21 social science experiments published in Nature and Science between 2010–2015. 13 of 21 (62%) replicated in the same direction with a significant effect. The average replication effect size was about 50% of the original. A prediction market among scientists correctly predicted replication outcomes about 75% of the time — suggesting the community has reasonable intuitions about which findings are real.

Method Snapshot

direct replications

Background

basic statistics

What struck me here was the author’s thought that good reviewers know whether the effect will be reproduced. Which, to be fair, makes you think that they don’t always say that out loud…

ES