This protocol presents the planned methods and procedures for a meta-review that explores the methodological quality of education meta-analyses and their potential significance for research use. An ever-increasing number of meta-analyses are published every year while at the same time new approaches to enhance the validity and reproducibility of systematic reviews are in constant development. We aim to conduct a meta-review to examine the current practices of education meta-analyses in terms of review procedures, meta-analysis methods, and strategies for making meta-analyses useful for non-research audiences. We will focus on meta-analyses including randomized controlled trials and quasi-experimental designs on the effects of K-12 school-based academic interventions on student academic achievement. A comprehensive search will be conducted to retrieve all studies that potentially meet our criteria. Study features will be coded to evaluate three main dimensions: the quality of the review process, which includes the practices used at each systematic review stage (e.g., search procedure, selection, and critical appraisal); the quality of the meta-analysis methods, including the methods used for synthesizing the results, exploring heterogeneity and additional analyses, such as publication bias assessment; the significance for research use, which includes evaluating the ways authors present findings to increase accessibility and relevance for practitioners (e.g., stakeholder engagement in the process, reporting of results). We will identify strengths and weaknesses of current meta-analysis practices and compare them across time, publication outlet, and funding status.

Protocol for a Meta-Review on Education Meta-Analyses: Exploring Methodological Quality and Potential Significance forResearch Use in Practice

Marta Pellegrini
;
2024-01-01

Abstract

This protocol presents the planned methods and procedures for a meta-review that explores the methodological quality of education meta-analyses and their potential significance for research use. An ever-increasing number of meta-analyses are published every year while at the same time new approaches to enhance the validity and reproducibility of systematic reviews are in constant development. We aim to conduct a meta-review to examine the current practices of education meta-analyses in terms of review procedures, meta-analysis methods, and strategies for making meta-analyses useful for non-research audiences. We will focus on meta-analyses including randomized controlled trials and quasi-experimental designs on the effects of K-12 school-based academic interventions on student academic achievement. A comprehensive search will be conducted to retrieve all studies that potentially meet our criteria. Study features will be coded to evaluate three main dimensions: the quality of the review process, which includes the practices used at each systematic review stage (e.g., search procedure, selection, and critical appraisal); the quality of the meta-analysis methods, including the methods used for synthesizing the results, exploring heterogeneity and additional analyses, such as publication bias assessment; the significance for research use, which includes evaluating the ways authors present findings to increase accessibility and relevance for practitioners (e.g., stakeholder engagement in the process, reporting of results). We will identify strengths and weaknesses of current meta-analysis practices and compare them across time, publication outlet, and funding status.
2024
Meta-analysis; Systematic review; Methodological quality; Transparency; Use of research evidence
File in questo prodotto:
File Dimensione Formato  
NJSRE_6169_Web.pdf

accesso aperto

Tipologia: versione editoriale (VoR)
Dimensione 150.56 kB
Formato Adobe PDF
150.56 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11584/394403
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact