For many years, the U.S. Department of Education’s Institute of Education Sciences (IES) has been tasked with answering a crucial question: What works in education? Specifically, which programs, interventions, and services help students learn? The What Works Clearinghouse (WWC) defines quality evaluations, including statistical methods that allow for confident assertions about a program’s effectiveness in improving student outcomes. While this focus represents a niche within the broader field of education research, IES and the WWC set the gold standard for evaluating educational programs and interventions, making this niche area highly influential.
In recent years, significant budget cuts by the Trump administration have led to layoffs and uncertainty in IES funding. This turmoil has affected top evaluation researchers in think tanks and policy shops, leading to job losses and a crisis in the field of education evaluation research. This crisis presents an opportunity for reinvention.
The current approach to understanding educational effectiveness is too narrow. It primarily focuses on determining causality and using readily available quantitative measures like test scores. This hyper-focus on causality often leads to a narrow view of education, where the study design dictates what is evaluated rather than the other way around. This approach overlooks the broader context and mechanisms that drive educational outcomes.
The evaluation field’s reliance on quasi-experimental studies often results in omitting crucial aspects. These studies frequently miss exploring mechanisms that explain why an intervention works in specific contexts. They also overlook nuanced outcomes that are difficult to quantify, such as critical thinking skills, socioemotional competencies, and psychological well-being. These outcomes, while challenging to measure, are vital for understanding whether students thrive.
To address these issues, a new approach is needed. This approach should include more causal research that broadens the lens, incorporating qualitative research that complements causal results. Emphasizing mechanisms, as seen in psychology, can help test hypotheses about why interventions work. Additionally, measures beyond achievement should be developed to capture a more comprehensive view of educational outcomes.
A new approach for IES and the WWC
This new approach can be applied to IES and its future endeavors. For example, publishing experimental or quasi-experimental results alongside qualitative or survey-based studies on the same intervention can provide a more holistic understanding. The quasi-experimental study would hypothesize and model the causal mechanism, while the qualitative study would validate this mechanism and consider contextual factors that might enhance or undermine its impact.
This approach would incentivize cross-disciplinary research, helping to explain which interventions are portable to other contexts. Being more attentive to context and conditions—understanding the “how” and “why” of what works in education—can address criticisms that education research is difficult to translate into concrete actions and decisions for policymakers and practitioners. By better pinpointing mechanisms, educators can determine if a strategy tried elsewhere is likely to benefit a particular group of students in a specific context facing particular challenges.
The IES and its endeavors, including the WWC, have played a vital role in education by identifying what works. However, pairing the WWC with a “how and why it works” clearinghouse could provide a more comprehensive understanding of educational effectiveness. This crisis in education evaluation research should not be wasted; it presents an opportunity to reinvent and broaden our approach to understanding educational effectiveness.
