Why Growth Mindset Meta Analyses Disagree So Much

Introduction

Growth mindset is a particularly revealing test case for evidence-based self-improvement because serious meta-analyses of broadly the same research literature have produced strikingly different messages. One concludes that interventions have, at most, a tiny average effect on academic achievement and that the apparent benefit disappears under stricter quality and publication-bias checks. Another argues that averaging everybody together answers the wrong question: well-implemented interventions targeted at students facing academic difficulty show a small but meaningful benefit.[PubMed]pubmed.ncbi.nlm.nih.govDo growth mindset interventions impact students' academic achievement? A systematic review and meta-analysis with recommendations f…

The disagreement is not simply “one side believes in growth mindset and the other does not”. It turns on consequential methodological choices: which studies and outcomes enter the dataset, how multiple results from the same study are handled, how study quality and publication bias are assessed, whether moderators are analysed separately or jointly, and whether researchers focus on a grand mean or on variation between populations and settings. That makes growth mindset unusually useful for understanding why a meta-analysis is not a mechanical truth machine.

Why overall effect estimates became controversial

A growth mindset, in this literature, means believing that intellectual ability can develop rather than being permanently fixed. The practical claim goes further: teaching this belief might change how students respond to difficulty and thereby improve achievement. The distinction matters because evidence that mindset correlates with achievement is not equivalent to evidence that an intervention changing mindset will improve grades.

An influential 2018 synthesis by Victoria Sisk and colleagues already suggested caution. Its first meta-analysis covered 273 samples and 365,915 participants and examined the association between mindset and achievement; its second covered 43 intervention studies and 57,155 participants. Overall relationships and intervention effects were weak, although some analyses suggested that students from lower socioeconomic backgrounds or those academically at risk might benefit more.[PubMed]pubmed.ncbi.nlm.nih.govTo What Extent and Under Which Circumstances Are Growth Mind-Sets Important to Academic Achievement? Two Meta-Analyses - PubMedMarc…

The argument sharpened with two systematic reviews published together in Psychological Bulletin in 2023.

Macnamara and Burgoyne examined 63 studies with 97,672 participants and estimated an overall academic-achievement effect of d = 0.05, with a 95% confidence interval from 0.02 to 0.09. That is a very small standardised difference. More importantly, their progressively stricter analyses made the result smaller rather than larger. Among 13 studies in which the intervention demonstrably changed students’ mindsets, the estimated effect was d = 0.04, with a confidence interval crossing zero. Restricting the analysis to six studies judged to provide the highest-quality evidence produced d = 0.02, again statistically non-significant.[PubMed]pubmed.ncbi.nlm.nih.govDo growth mindset interventions impact students' academic achievement? A systematic review and meta-analysis with recommendations f…

Burnette and colleagues approached the literature differently. Rather than treating heterogeneity mainly as noise around a single average, they asked whether effects differed in theoretically predicted ways. Their review covered 53 independent samples across academic, mental-health and social outcomes. For academic achievement, their broader estimate was about d = 0.09; when analyses focused on targeted populations expected to benefit and interventions delivered with high fidelity, the estimate was d = 0.14, 95% CI 0.06 to 0.22.[PubMed]pubmed.ncbi.nlm.nih.govA systematic review and meta-analysis of growth mindset interventions: For whom, how, and why might such interventions work? - PubMed…

Those numbers are all small, but they imply different conclusions. An effect near 0.02–0.05 supports scepticism about using growth mindset as a general achievement intervention. An effect around 0.14 in a properly specified target population supports a narrower proposition: a cheap intervention might be worthwhile for some students in some circumstances.

The key question therefore became less “Which meta-analysis has the correct headline number?” than what population and intervention does each number actually describe?

The same literature can become different datasets

Meta-analysis involves many decisions before a pooled effect is calculated. Researchers define what counts as a growth mindset intervention, which designs qualify, which achievement measures count, whether unpublished work enters the synthesis and how multiple outcomes or subgroups within one experiment are represented. Different defensible rules can therefore produce datasets that overlap substantially without being identical.

More importantly, the two 2023 reviews handled the structure of those data differently. Tipton and colleagues’ subsequent methodological commentary argued that Macnamara and Burgoyne aggregated effect sizes within studies before pooling them and tested moderators largely through separate subgroup analyses. Burnette and colleagues instead retained more within-study variation and used multilevel meta-regression, allowing related effects from different outcomes and subgroups to be modelled without pretending they were independent studies.[PubMed]pubmed.ncbi.nlm.nih.govOpen source on nih.gov.

That technical difference matters when the theory itself predicts heterogeneity. Imagine an intervention has essentially no effect on students already thriving academically but a modest effect on students struggling with setbacks. Average the groups first and the targeted effect is diluted. Preserve subgroup variation and model risk status, and the difference may become visible.

Tipton and colleagues tested precisely this issue by applying methods similar to Burnette’s multilevel approach to Macnamara and Burgoyne’s dataset. Their exploratory re-analysis again found substantial heterogeneity and a significant effect for focal, at-risk groups, including after adjustment for measures of study quality or publication bias. They therefore argued that the disagreement could not be reduced simply to the two teams having selected different studies.[PubMed]pubmed.ncbi.nlm.nih.govOpen source on nih.gov.

Macnamara and Burgoyne rejected the implication that modelling heterogeneity resolved the deeper evidential problems. Their reply stressed that they had preregistered numerous moderator tests and argued that methodological weaknesses and bias in the underlying studies remained consequential regardless of the statistical model. They reported, among other concerns, that 94% of interventions contained what they classified as confounds and that researchers with a known financial incentive were substantially more likely to report positive results.[EBSCO OpenURL]openurl.ebsco.comOpen source on ebsco.com.

So the dispute exposes two different risks. Over-aggregation can conceal a real conditional effect; subgroup modelling can identify effects that matter only in particular circumstances. But increasingly elaborate moderator analyses cannot rescue an underlying evidence base if its positive results are substantially driven by weak designs, selective reporting or researcher degrees of freedom. Both problems have to be considered.

How study quality and publication bias change the result

The strongest sceptical argument is not merely that the average effect is small. It is that estimated effects appear to shrink as the evidence is filtered for credibility.

Macnamara and Burgoyne reported shortcomings involving intervention design, analysis and reporting across the literature. They also found evidence consistent with publication and researcher bias. Their uncorrected estimate across all studies was d = 0.05; after statistical adjustment for possible publication bias, their estimate fell to roughly d = 0.03 and was no longer statistically significant. Their highest-quality subset yielded d = 0.02.[PubMed]pubmed.ncbi.nlm.nih.govDo growth mindset interventions impact students' academic achievement? A systematic review and meta-analysis with recommendations f…

Publication-bias corrections are themselves model-dependent and should not be treated as definitive measurements of the “true” effect. The broader lesson is more robust: when an intervention’s estimated benefit is only a few hundredths of a standard deviation, modest amounts of selective reporting, attrition, analytical flexibility or design weakness can determine whether the final headline says “effective” or “no reliable effect”.

The quality-filtering argument has since received further support. A 2025 structured review by Carolina Gazmuri restricted its attention to randomised controlled trials of intelligence-focused growth mindset interventions for school-age pupils with academic-performance outcomes. It found that the most secure studies produced effects clustered close to zero, with the highest-rated evidence ranging from approximately d = −0.01 to.065. The review concluded that schools should not devote substantial resources to growth mindset programmes in expectation of meaningful academic gains, although very cheap implementations might still be reasonable given the possibility of a small positive effect.[Bera Journals]bera-journals.onlinelibrary.wiley.comBera JournalsCan growth mindset interventions improve academic achievement? A structured review of the existing evidence - Gazmuri - 2025…

That review also illustrates why “high quality” is not a perfectly objective switch. Gazmuri criticised aspects of Macnamara and Burgoyne’s own quality filter: none of the studies met all their best-practice criteria, so their highest-quality category used a threshold, and some studies admitted to it still had substantial attrition.[Bera Journals]bera-journals.onlinelibrary.wiley.comBera JournalsCan growth mindset interventions improve academic achievement? A structured review of the existing evidence - Gazmuri - 2025…

Quality assessment therefore introduces judgement rather than eliminating it. One review may emphasise randomisation, attrition and independent evaluation; another may place additional weight on whether the intervention actually changed the proposed psychological mediator; another may penalise confounded intervention components. A result becoming smaller under a quality filter is informative, but readers still need to ask what that filter measured.

Targeted effects are plausible, but they narrow the claim

The strongest evidence for the heterogeneity argument comes from experiments designed around it rather than from retrospective subgroup searches.

The 2019 National Study of Learning Mindsets randomly tested a brief online intervention in a nationally representative sample of US secondary schools. Its primary preregistered analysis focused on 6,320 relatively lower-achieving students. Among those students, the intervention shifted mindset substantially and increased core-course GPA by 0.10 grade points, corresponding to a standardised effect of 0.11. The study used independent data collection and processing, preregistration and a separate blinded Bayesian analysis, features intended to reduce several familiar sources of researcher bias.[Nature]nature.comOpen source on nature.com.

Just as importantly, the effect was not constant between schools. Benefits were larger where peer norms supported taking on academic challenges, and they varied with school achievement level. The study therefore provides a concrete example of what the heterogeneity argument means: changing a student’s interpretation of difficulty may accomplish little if the surrounding environment does not provide opportunities or social support for acting on that interpretation.[Nature]nature.comOpen source on nature.com.

Yet this evidence does not vindicate the broad popular version of growth mindset. It narrows it. “Teaching anyone a growth mindset improves their performance” is a much stronger proposition than “a carefully developed, brief intervention may produce small academic benefits for some lower-achieving adolescents in environments that support the behavioural response it is intended to encourage”.

There is another statistical danger here. Once the overall effect is small, it becomes tempting to search repeatedly for subgroups in which it is larger. Enough exploratory slicing can produce apparently impressive moderators by chance. The credibility of a targeted claim therefore depends heavily on whether the subgroup was predicted in advance, whether the moderator is theoretically coherent, whether the analysis has enough statistical power and whether the pattern recurs in independent studies.

This is why the disagreement between the reviews cannot sensibly be resolved by saying either “subgroups count” or “only the overall average counts”. A preregistered effect in a prespecified target group is stronger evidence than an attractive subgroup discovered after looking at the results. Conversely, an overall average can be a poor description of an intervention explicitly intended for a narrower population.

Prediction intervals change what “it works” means

Confidence intervals and prediction intervals answer different questions, and growth mindset provides a useful demonstration of the difference.

A confidence interval around a meta-analytic mean indicates uncertainty about the average effect. A prediction interval goes further: it estimates the range in which the true effect of a comparable future study might plausibly fall, given the observed between-study variation.

Burnette and colleagues estimated d = 0.14 for academic achievement when focusing on targeted samples and high-fidelity implementation, but their 95% prediction interval ran from −0.08 to 0.35.[PubMed]pubmed.ncbi.nlm.nih.govA systematic review and meta-analysis of growth mindset interventions: For whom, how, and why might such interventions work? - PubMed…

That interval is arguably more useful for a self-improvement claim than the mean alone. The average is positive, yet the prediction interval includes zero and negative effects. In plain terms, even if the conditional average is accepted, the research does not imply that a new implementation should reliably reproduce a benefit of 0.14 standard deviations. Effects can differ substantially with participants, implementation and context.

Tipton and colleagues made this variation central to their critique. They argued that when an intervention literature is genuinely heterogeneous, meta-analysis should characterise the distribution of effects, not merely announce whether its pooled mean passes a significance threshold. Their commentary noted that Burnette’s academic prediction interval explicitly showed outcomes ranging from mildly negative to considerably more positive than the mean.[PubMed Central (PMC)]pmc.ncbi.nlm.nih.govOpen source on nih.gov.

This produces an important shift in interpretation. A confidence interval can support the sentence “the estimated average effect in this specified population is positive”. A prediction interval can simultaneously warn that “positive benefit is not guaranteed in the next setting”. Those statements are not contradictory.

What the competing meta-analyses actually justify

For readers interested in self-improvement that works, the growth mindset literature supports a more modest conclusion than either the strongest promotional or dismissive versions of the idea.

There is little basis for treating growth mindset as a general-purpose route to substantially higher achievement. Large reviews put the unrestricted average effect at roughly 0.05 to 0.09 standard deviations, and analyses placing greater weight on study quality have produced estimates close to zero. The 2025 structured review strengthens the case that the most methodologically secure school trials generally show null or very small academic effects.[PubMed]pubmed.ncbi.nlm.nih.govDo growth mindset interventions impact students' academic achievement? A systematic review and meta-analysis with recommendations f…

At the same time, “the grand mean is tiny” does not prove that every intervention in every population is ineffective. Burnette and colleagues’ analysis, Tipton and colleagues’ re-analysis, and the National Study of Learning Mindsets all provide reasons to take targeted and contextual effects seriously. The best-supported version of that argument concerns students confronting academic difficulty and interventions implemented in conditions capable of supporting the behaviour the message is supposed to encourage.[nih.gov]pubmed.ncbi.nlm.nih.govA systematic review and meta-analysis of growth mindset interventions: For whom, how, and why might such interventions work? - PubMed…

The resulting evidence hierarchy is more useful than a binary verdict. A claim that simply having a growth mindset reliably transforms achievement is poorly supported. A claim that brief mindset interventions produce large academic improvements across populations is also inconsistent with the strongest quantitative reviews. A narrower claim — that particular, well-designed interventions can sometimes produce small benefits for appropriately targeted students, with outcomes depending on implementation and context — remains compatible with the evidence, but its expected effect is modest and uncertain.

Most importantly, the controversy shows why two published meta-analyses can inspect much of the same literature without producing interchangeable answers. The pooled estimate depends on what gets pooled. Study-quality filters can reduce an apparent effect; publication-bias assumptions can change statistical significance; averaging within studies can hide subgroup variation; multilevel modelling can recover that variation; moderator choices can either illuminate genuine conditional effects or create opportunities for overinterpretation; and prediction intervals can reveal uncertainty concealed by a positive mean.

Growth mindset is therefore less convincing as a universal self-improvement prescription than as a lesson in reading behavioural evidence carefully. The meaningful question is rarely just “Does it work?” It is how large is the best-supported effect, for whom, under which conditions, how variable is it across settings, and does that conclusion survive when the least trustworthy evidence is given less weight?

Amazon book picks

Further Reading

Books and field guides related to Why Growth Mindset Meta Analyses Disagree So Much. Use these as the next step if you want deeper reading beyond the article.

Book

Mindset

By Carol S. Dweck

Rating: 4.5/5 from 11 Google Books ratings

From the renowned psychologist who introduced the world to “growth mindset” comes this updated edition of the million-copy bestseller—fea...

See on Amazon

Book

The Art of Statistics

By David Spiegelhalter

'A statistical national treasure' Jeremy Vine, BBC Radio 2 'Required reading for all politicians, journalists, medics and anyone who trie...

See on Amazon

Book

Calling Bullshit

By Carl T. Bergstrom, Jevin D. West

Bullshit isn’t what it used to be. Now, two science professors give us the tools to dismantle misinformation and think clearly in a world...

See on Amazon

Book

The Data Detective

By Tim Harford

From “one of the great (greatest?) contemporary popular writers on economics” (Tyler Cowen) comes a smart, lively, and encouraging rethin...

See on Amazon

Browse more on Amazon:MindsetThe Art of StatisticsCalling Bullshit

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Live-tested eBay searches with available results related to this page.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Selected frompsychology poster oneBay.co.uk.

Current eBay listing

Reverse Psychology Quote Framed Wall Art Poster Canvas Print Picture

GBP 8.99 | Shipping GBP 4.99NewSeller: mirrashop2018 (99.8% positive)Ships from: GB

View listing on eBay

Current eBay listing

Psychologist Psychology Framed Wall Art Poster Canvas Print Picture

GBP 29.95 | Shipping GBP 4.99NewSeller: jjestoreltd (99.6% positive)Ships from: GB

View listing on eBay

Current eBay listing

Political Psychology Framed Art Pri Framed Wall Art Poster Canvas Print Picture

GBP 34.95 | Shipping GBP 4.99NewSeller: jjestoreltd (99.6% positive)Ships from: GB

View listing on eBay

Current eBay listing

Psychology A4 Poster Print Lab Rat, Classical Conditioning Art for Psychologist

GBP 9.99 | Free shippingNewSeller: treasured words (99.6% positive)Ships from: GB

View listing on eBay

Browse more oneBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: openurl.ebsco.com
Link:https://openurl.ebsco.com/contentitem/doi%3A10.1037%2Fbul0000394?id=ebsco%3Adoi%3A10.1037%2Fbul0000394&sid=ebsco%3Aplink%3Acrawler

2. Source: caslabs.case.edu
Title: CAS Labs Publications – Brooke N. Macnamara Laboratories
Link:https://caslabs.case.edu/macnamaralabs/publications/

3. Source: nature.com
Link:https://www.nature.com/articles/s41586-019-1466-y

4. Source: openurl.ebsco.com
Link:https://openurl.ebsco.com/contentitem/doi%3A10.1037%2Fbul0000370?id=ebsco%3Adoi%3A10.1037%2Fbul0000370&sid=ebsco%3Aplink%3Acrawler

5. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/36326645/

6. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/36227318/

7. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/29505339/

8. Source: bera-journals.onlinelibrary.wiley.com
Link:https://bera-journals.onlinelibrary.wiley.com/doi/10.1002/rev3.70066

9. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/37701627/

10. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC10495100/

11. Source: bera-journals.onlinelibrary.wiley.com
Link:https://bera-journals.onlinelibrary.wiley.com/doi/abs/10.1002/rev3.70066

12. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/31391586/

13. Source: doi.org
Link:https://doi.org/10.1080/01443410.2025.2553639

14. Source: bera-journals.onlinelibrary.wiley.com
Link:https://bera-journals.onlinelibrary.wiley.com/toc/20496613/2025/13/2

15. Source: ifp.nyu.edu
Title: rev3 70066
Link:https://ifp.nyu.edu/2025/journal-article-abstracts/rev3-70066/

16. Source: bera-journals.onlinelibrary.wiley.com
Link:https://bera-journals.onlinelibrary.wiley.com/doi/epdf/10.1002/rev3.70066

17. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/40086228/

18. Source: asmepublications.onlinelibrary.wiley.com
Link:https://asmepublications.onlinelibrary.wiley.com/doi/full/10.1111/medu.15391

19. Source: spssi.onlinelibrary.wiley.com
Link:https://spssi.onlinelibrary.wiley.com/doi/full/10.1111/asap.12367

20. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC10299668/

21. Source: compass.onlinelibrary.wiley.com
Link:https://compass.onlinelibrary.wiley.com/doi/abs/10.1111/spc3.12723

22. Source: doi.org
Link:https://doi.org/10.1111/bjep.12572

23. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC8299535/

24. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/31494041/

25. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/31391586/?dopt=Abstract

26. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC6786290/

Additional References

27. Source: psychologicalscience.org
Link:https://www.psychologicalscience.org/journals/psychological-science/0956797617739704/

28. Source: youtube.com
Link:https://www.youtube.com/watch?v=BNMHc2Pefl4

29. Source: youtube.com
Title: The power of believing that you can improve | Carol Dweck | TED
Link:https://www.youtube.com/watch?v=_X0mgOOSpLU

30. Source: youtube.com
Title: Growth Mindsets & Academic Improvement
Link:https://www.youtube.com/watch?v=whl9HdENupQ

31. Source: youtube.com
Title: Developing a Growth Mindset with Carol Dweck
Link:https://www.youtube.com/watch?v=hiiEeMN7vbQ

32. Source: scispace.com
Link:https://scispace.com/papers/do-growth-mindset-interventions-impact-students-academic-ggnnbcyw

33. Source: scispace.com
Link:https://scispace.com/papers/do-growth-mindset-interventions-impact-students-academic-vdj4e7i0

34. Source: researchgate.net
Link:https://www.researchgate.net/publication/372427209_A_Spotlight_on_Bias_in_the_Growth_Mindset_Intervention_Literature_A_Reply_to_Commentaries_That_Contextualize_the_Discussion_and_Illustrate_the_Conclusion

35. Source: researchgate.net
Link:https://www.researchgate.net/publication/364370114_A_systematic_review_and_meta-analysis_of_growth_mindset_interventions_For_whom_how_and_why_might_such_interventions_work

36. Source: researchgate.net
Link:https://www.researchgate.net/publication/363698241_CULTURAL_FLUENCY_AND_SCIENTIFIC_PROGRESS_1_Culturally_Fluent_Theories_Metascience_and_Scientific_Progress_A_Case_Example_IN_PRESS_Psychological_Bulletin