How good is the evidence that sleeping on a problem helps?
I am trying to work out what conditions make one particular person good at their work. That means leaning on sleep and attention research, which means checking it first. It is often thinner than the confidence built on top of it.
Evidence In 2004 a study in Nature reported that sleeping on a problem helped people find a hidden rule. 59% of the sleep group found it. About 23% of the groups who stayed awake did.
That is the study behind every article you have read about sleeping on it.
The design had 106 people across five groups. 22 people in the group that produced the headline. One statistical test across all five groups, at p = .03. No effect size anywhere in the paper. No confidence interval.
Evidence What came after is the part that does not travel with the finding. The same group’s later work produced post-sleep rates of 31% and 19%, which overlaps the original study’s no-sleep baseline. A separate 77-person study found no effect at all on classic insight problems, with effect sizes of .003. A preregistered study in 2025 failed to replicate the fashionable follow-on claim about sleep onset.
Evidence It has been repeated, and this is where I got it wrong the first time I wrote this. I said nobody had run the task again. A 2019 systematic review of this exact task reports that almost all studies using it found sleep helping people state the hidden rule. The direction holds.
Evidence The size does not. A 2023 study ran the same task across sleep and wake. After a night of quiet sleep, 18% found the rule. Four people out of 22. Nobody in the wake group found it at all.
So the honest version is not that the finding failed. It is that the finding shrank. 59% became 18%, and the studies that found nothing at all were using different puzzles, the nine-dot and matchstick kind. Which points at something narrower and more interesting than the headline. Sleep may do something for this one task, and there is little sign it does anything for insight in general.
I am leaving my mistake in rather than deleting it. I searched, found no direct replication, and wrote that none existed. Two papers say otherwise. Not finding something is not evidence that it is not there, which is the rule I broke.
Where this stops. Wagner et al. 2004 in Nature, counts reprinted openly by the authors at PMC3902672. The 2019 systematic review is PMC6779511. The 2023 study is PMC10722168, which was testing pink noise during sleep and reported the silent-sleep and wake groups alongside it. The nulls on other puzzle types are Schonauer et al. 2018 at PMC5834438 and Brodt et al. 2018. The preregistered failure on the sleep-onset claim is Lowe et al. 2025 at PMC12200826.
What I still cannot tell you. Whether anyone has run a registered direct replication of the original design. I have not found one, and after being wrong once I am not going to claim there is none.
The claim I am not making. That sleep does nothing. Sleep and memory is a real literature. It is the leap from this task to insight in general that does not hold.