A little meta science, if you will
On what research did I work over the last few years?
Editor’s note: I updated this post 5 April 2026 to provide a clearer report of the results from the SCORE project.—JohnL
On 1 April 2026,1 Nature published three articles about replication of research findings. The articles are reports of an effort to study the trustworthiness of scientific findings. These reports are important for special education, because our efforts to employ evidence- or research-based interventions are, obviously, grounded on results of studies of those interventions. If we want to employ evidence-based practices, we need to have assurance that the research base is trustworthy.
The three studies (and some other companion reports) examined aspects of replication. Readers may have heard about “the replication crisis”; these studies are direct examinations of aspects of that crisis. They are part of a substantial research project called Systematizing Confidence in Open Research and Evidence, They report the results of studies addressing these aspects of replication: reproducibility, robustness, and replicability.
Reproducibility
The SCORE project described reproducibility as follows:
The reproducibility study examined whether original findings can be recreated using the same data and analyses. Drawing on a stratified random sample of 600 papers, reproducibility assessments were conducted for papers in which data were publicly available or successfully obtained from authors.
The report of the reproducibility study was published under the title , “Investigating the reproducibility of the social and behavioural sciences” (Miske et al., 2026).
Robustness
The SCORE project described Robustness in this way:
The robustness study investigated the degree to which research findings depend on analysts’ choices. Independent re-analysts each reanalyzed the same dataset for 100 selected claims, allowing the program to assess the extent of analytical variability and its implications for scientific conclusions.
The report of the robustness project was published as “Investigating the analytical robustness of the social and Behavioural sciences” (Aczel et al., 2026).
Replicability
Here is the SCORE description of the replicability study:
The replicability study tested whether original positive findings generalize to new data, using high-powered replication attempts of 274 claims drawn from 164 papers.
The results of the replicability study were reported as “Investigating the replicability of the social and behavioural sciences” (Tyner et al., 2026).
I had the good fortune to work on two of these studies, the reproducibility and replicability projects. If one diligently searches, one can find my name in the long lists of authors. As did many co-authors, I did real work on them, but I’m just one of many ants who moved the grains of sand around to build the hill.
Results
In the robustness study, about 1/3rd of the reanalyses yielded identical results, but most yielded the same conclusion. The replicability study showed that about half of the claims found the same statistical pattern of results, although often of lower magnitude. The reproducibility study showed that about “half of the evaluated claims were precisely reproduced and that roughly three-quarters were at least approximately reproduced,” especially in political science and economics, more recent studies and those from studies that required sharing of original data. SCORE provided a clear summary of the outcomes; I drew some of this summary from that source.
Press coverage
Journalists whose beat is science have published articles about the research. One can read what they wrote, so you won’t have to take my word for it. Here are links to a few examples of their coverage:
Nicola Jones of Nature had a news story 1 April 2026 about the work, too. It’s headlined, “Half of social-science studies fail replication test in years-long project: Results from massive, ‘eagerly awaited’ initiative reinforce concerns about the credibility of science — but raise hope for solutions.”
Jeffrey Brainard of Science published (1 April 2026) “Across the social sciences, half of research doesn’t replicate: Ambitious effort tested whether more than 100 papers held up on multiple types of ‘repeatability’ tests.
Carl Zimmer of the New York Times published “Can Science Predict When a Study Won’t Hold Up?: Conducting research is hard; confirming the results is, too. And artificial intelligence isn’t yet ready to help, a major new study finds” on 1 April 2026.
SCORE is an example of research about research for researchers, a cottage niche in the research world that researchers often call “metascienc” or “meta-research.” There’s a little “inside baseball” and “in the weeds” about this sort of research. The topic is a bit geeky, but the Wikipedia entry on metascience provides a reasonably accurate and informative explanation.
References
Aczel, B., Szaszi, B., Clelland, H. T., Kovacs, M., Holzmeister, F., Van Ravenzwaaij, D., ... & Geraldes, D. (2026). Investigating the analytical robustness of the social and behavioural sciences. Nature, 652(8108), 135-142. https://www.nature.com/articles/s41586-025-09844-9.pdf
Miske, O., Abatayo, A. L., Daley, M., Dirzo, M., Fox, N., Haber, N., ... & Errington, T. M. (2026). Investigating the reproducibility of the social and behavioural sciences. Nature, 652(8108), 126-134. https://www.nature.com/articles/s41586-026-10203-5
Tyner, A. H., Abatayo, A. L., Daley, M., Field, S., Fox, N., Haber, N. A., ... & McHugh, C. (2026). Investigating the replicability of the social and behavioural sciences. Nature, 652(8108), 143-150. https://www.nature.com/articles/s41586-025-10078-y
Footnote
No joking! As far as I know, that date was’t selected for the purpose of making fun of the