Learning Losses During the Pandemic: Reviewing the Evidence

Considerable evidence has been suggesting for some time now that children suffered learning loss from the K-12 during the pandemic.

From a research perspective, it’s trickier to demonstrate exactly why children suffered such losses. School closures are an obvious possibility, and probably also part of the answer. But the other disruptions of the pandemic to family health, economic prospects, and social lives surely played a complementary role. But for the affected children, the arguments over causes are also arguments about the past: for this group, the policy question is whether remediation efforts can help them catch up to where they would have been.

Santiago Pinto pulls together the existing research on this issue in “The Pandemic’s Effects on Children’s Education” (Economic Brief: Federal Reserve Bank of Richmond, August 2023, #23-29).

As I’ve noted here a couple of months ago, recent test scores show students falling behind during the pandemic. Pinto writes: “The 2023 paper “A Systematic Review and Meta-Analysis of the Evidence on Learning During the COVID-19 Pandemic” looked at more than 40 studies around the world and reached similar conclusions. In the study, the overall size effect is a loss of 0.14 SD [standard deviations], or about 0.4 years of schooling lost.`` Evidence from the US shows losses of this size or slightly larger.

It’s worth putting that learning loss of about a half-year in perspective. In some ways, it doesn’t seem like a lot. Shake it off! Catch up! But US schools have had a substantial share of students lagging behind grade-level for decades now, and they have not shown much ability to help large groups of students catch up. Indeed, if there was a school reform that would boost average student learning by a half-year, it would be thought of as a near-miracle. Conversely, losing a half-year of schooling is a near-disaster.

Moreover, US schools do not seem to be mobilizing in a way that will help students catch up. For example, students have shifted out of K-12 schools to alternatives and absenteeism is high. Pinto writes:

The 2023 report “Where the Kids Went: Nonpublic Schooling and Demographic Change During the Pandemic Exodus From Public Schools” uses data from 21 states plus D.C. (covering the 2020-21 and 2021-22 school years) to explain what happened with students leaving public schools. The report shows that an increase in home-school enrollment and a decrease in school-age population each explain 26 percent of the decline in public-school enrollment, and 14 percent was explained by an increase in private-school enrollment. About one-third of the decline, however, remains unexplained.

We don’t yet know if those who shifted to alternative schools will be more able to catch up. We do know that school districts with less in-person learning had the biggest drops in test scores, which doesn’t suggest that the alternatives are performing especially well. In general, student learning is tied to the amount and the continuity of instructional time received, and neither the amount nor the continuity seems to be rising in a way that will help students as a group to catch up.

It’s of course dicey to estimate what learning loss means in the long-run. Pinto describes one effort this way:

[A]n additional year of schooling increases income by about 11 percent on average in the U.S. If it is assumed that the pandemic and the associated changes in schooling generated a loss of one-third of a school year, then this would translate into a loss of income for the affected students of about 3.5 percent over their entire working life. 

The losses in lifetime earnings are of course only part of the issue: “Empirical evidence has linked school closures to several factors, including rising mental health concerns, lower levels of engagement, reports of violence against children, rising obesity, increases in teenage pregnancy, rising levels of chronic absenteeism and dropouts, and overall deficits in the development of socioemotional skills due to social isolation from networks and peers.”

These effects tend to be larger for students who are from economically disadvantaged families, and those who are already falling behind academically. Schools tend to be a “great equalizer,” making up to some extent for the unequal distribution of other educational resources across families. Those children who depended most on public schools also lost the most when the schools shut down.

What would an emergency-level response to these issues look like? One possible step would be a dramatic rise in programs of in-person tutoring. “The 2022 working paper “The Challenges of Implementing Academic COVID Recovery Interventions: Evidence From the Road to Recovery Project” claims that making up the gap would require approximately 40 to 100 hours of high-quality tutoring for the average student (slightly less for reading than math).” Think of this as maybe two hours of tutoring per student every week. Making this happen would require an immediate and dramatic expansion of tutoring programs, drawing on parents, retirees, college students , and others.

Another step would be to expand the school year by, say, six weeks: “The 2023 paper “The Fiscal and Welfare Effects of Policy Responses to the COVID-19 School Closures” shows that extending schools by three months (or six weeks over the next two summers) generates significant welfare gains for the children and raises future taxes to pay for the cost of this schooling expansion.”

Frankly, a dramatic rise in tutoring and expanded school years should have been much higher on the K-12 policy agenda before the pandemic.  During the pandemic, schools should already have been planning for the implementation of these steps back in 2021 and 2022. A decrepit bridge can sometimes wait a few years for necessary repairs. A current fourth-grader or an eight-grader or a high school senior, lagging behind academically, doesn’t have the luxury of waiting.