Pandemic Learning Loss: Why So Little Done About It?

Some problems can wait a few years.

A road is wearing out? You can patch it now, and fix it more thoroughly one of these years. But the learning loss experienced by K-12 students during the pandemic (for earlier discussions, see here and here) isn’t in the delayable category. The pandemic hit in February 2020. With the ’23-24 school year underway, the ’21-’22 school year and the ’22-’23 school years are now behind us. The processes for catching up need to already be underway, but they aren’t.

Jonathan Guryan and Jens Ludwig discuss the situation in “Overcoming Pandemic-Induced Learning Loss” (forthcoming in Building a More Resilient US Economy, edited by Melissa S. Kearney, Justin Schardin, and Luke Pardue from the Aspen Economic Strategy Institute, due out November 9, 2023, pp. 150-170). They describe the challenge of learning loss in this way:

For example, data from a single week in May 2020 showed that nearly a third of the Chicago Public School system’s 350,000 students did not log on to even one Google Classroom or Google Meet (Chicago Public Schools 2020; n.d.). Chronic absenteeism increased dramatically across the country, with student absences fully doubling in high-remote-instruction states like Virginia and California. (Given data limitations, those figures may, if anything, even underestimate the true rise in absenteeism). The US Department of Education estimated that at least 10.1 million students missed at least 10 percent of the 2020–2021 school year (Chang, Balfaz, and Byrnes 2022). Of course, missing this much school, and the imperfect substitution of remote school for in-person instruction, led to large learning losses, particularly for the most disadvantaged children in America. But the real public policy challenge is not merely short-term learning losses. Because education is intrinsically cumulative, there is the real possibility that pandemic-induced school disruptions may set a whole generation of students off track for the rest of their lives. …

The consequences of pandemic-induced learning loss, in other words, are likely to be long-term, and these consequences will be most dire for the most disadvantaged children. The potential magnitude of the long-term effects can be seen by pre-pandemic data on what happens when children miss key developmental milestones. Students who can’t read at grade level by third grade are four times less likely to graduate high school. Ninth graders who have not yet passed their required entry-level math class (Algebra I) are five times less likely to graduate.

Anyone consuming news in 21st-century America should be developing sensitivity to exaggeration, and this “rest of their lives” rhetoric may feel overstated. But in this case, for many students, it’s not. Making or missing these kinds of landmarks like high school graduation really does affect lifetime prospects.

A standard policy proposal for helping students catch up is known as “high-dosage tutoring,” which refers to having students who are falling behind meet with a tutor in a small group once or twice a week. Indeed, when Congress passed the Elementary and Secondary School Emergency Relief (ESSER) Fund legislation in 2020 and 2021 sending about $190 billion to K-12 schools to address pandemic-related expenses, one recommendation was that they earmark some of the funds for tutoring programs.

Guryan and Ludwig review some evidence on the effectiveness of tutoring. For example:

A series of demonstration projects in the 1980s found that compared to regular classroom instruction, students tutored one-to-one spend almost 40 percent more time on-task. Students in tutoring learned fully 2 standard deviations (SDs) more than their peers in traditional classroom settings (Bloom 1984). As a way to benchmark the enormous magnitude of that learning gain, the average test-score gain over the course of a student’s high school career is about 0.6–0.7 SDs, and the test-score gap between high- and low-income eighth graders is 1.4 SDs (Reardon 2011; Loveless 2012). Another way to get a sense of the magnitude here is that a student who improved their test score by 2 SDs would move approximately from the 15th to the 85th percentile. We also see large gains from tutoring outside of controlled lab conditions, in real-world school settings. A review of more than 90 randomized controlled trials (RCTs) of smaller-scale tutoring programs showed an average effect of 0.37 SDs (Nickow, Oreopoulos, and Quan 2020).

As they note, high-dosage tutoring “is plausibly the intervention most up to the task of meeting the scale of our current learning-loss challenge.” So why hasn’t it happened?

The reasons are so pedestrian as to be tragic. Tutoring would require bringing in outsiders to do the tutoring. Remember that we’re talking here about tutoring very basic math and reading skills, so finding people who could be certified to do the job on a part-time basis, from retirees to parents to college students, doesn’t seem to be an impossible task. But it would require institutional flexibility and rescheduling time, because the most effective tutoring programs happen during the usual school day, rather than requiring students to come early or stay late. As Guryan and Ludwig write:

Presumably, that’s been hard for schools to do in part because all organizations suffer from a general change-aversion. … What we have seen in practice is that when schools are faced with the possibility of change, they tend to do fewer of the hard things that will help students and more of the easier things that are likely to have fewer learning benefits for children. For example, in our experiences working with districts around the country, many have punted on the problem of trying to find time during the school day and instead relied on after-school programs or tried virtual tutoring at home in the evenings or on weekends. None of those efforts that we have seen firsthand led to a high “dosage” of tutoring delivered to students at any sort of scale. As another example, a different district we worked with tried a decentralized approach to tutoring, giving individual schools lots of discretion over how they deployed their tutors. Often, the tutors wound up simply serving as teachers’ aides, which the research suggests have little impact on student learning in part because these aides wind up being assigned to largely do the parts of the teacher’s job teachers like least (grading, making copies, etc.) …

Guryan and Ludwig argue for additional federal funding to support tutoring programs. I’m not definitively opposed to federal funding here, but financing of K-12 education has traditionally been a state responsibility. I’d be happier if states would show a commitment to high-dosage tutoring by setting up systems to certify tutors and to reschedule school days, using a combination of the existing federal money and their own funds. But this seems like a situation where, after the pandemic, K-12 schools just wanted to go back to doing what they had before the pandemic.