Most school districts experienced declines, but the extent varied.
Those are the findings of more than half a dozen studies published in recent months that examined the pandemic’s toll on academic performance. Across the board, they find big drops between spring 2019, before the pandemic hit, and spring 2021, a year after.
“The pandemic was like a band of tornadoes that left devastating learning losses in some districts and left many other districts untouched,” said Tom Kane, faculty director of the Center for Education Policy Research at Harvard University.
Students made more progress last year, but it was nowhere near enough to make up for the losses already incurred.
“People were hoping, ‘Oh my gosh, there’s going to be a lot of natural backlash,’ and we didn’t see that last year,” Kane said. “Maybe it will happen this year, but I’m not sure there’s much evidence for that hope.”
The high price of distance learning
One of the fiercest debates in the first year of the pandemic was how quickly schools should reopen and how significant the consequences would be of keeping them closed. We now have some answers.
A body of evidence charts setbacks that were more severe the longer students stayed in the virtual school. These studies examined the impact of personal vs. distance learning in the 2020-21 school year, where policies varied widely. In Texas and Florida, Republican governors ordered schools to operate in person starting in the fall of 2020. Elsewhere, and often in big cities, resistance and fear of the virus among teachers and parents kept schools virtual for a year or longer.
Different studies rely on different data sets and describe the extent of the impact to varying degrees, but they all point in the same direction:
· A study that uses data from testing firm NWEA found modest academic declines for students who quickly returned to in-person classes in the fall of 2020. But achievement losses were far higher for those who learned from home, and they were most pronounced for students in high-poverty, mostly remote schools, thus widening long-standing racial and economic divides.
Students who were full-time in-person during 2020-21 lost an average of 7.7 weeks of learning in maths. But those who were in a virtual classroom for more than half the year lost more than double that — an average of 19.8 weeks.
This research was based on NWEA assessments of 2.1 million students in 10,000 districts and analyzed by researchers at NWEA, Harvard and the American Institutes for Research.
· An Ohio study found that reading achievement in school districts that went all-out fell on average two or three times as much as it did for those who studied in person during the 2020-21 school year.
It looked closely at third graders because these students take reading tests in the fall and spring so that growth over the course of a school year can be assessed. During the 2020-21 school year, those learning remotely fell twice as far behind as those learning in person, compared to what would be expected in a pre-pandemic year.
“The more weeks of distance learning, the less students learned during that period,” said Vladimir Kogan, a political scientist at Ohio State University who produced those reports.
For math, the relationship in the Ohio data was less clear, with the declines most severe for students whose districts used a mix, or hybrid, of in-person and distance learning.
· A study of state test scores in 11 states by Brown economist Emily Oster and others, districts with full personalized learning found smaller declines than those that operated externally, with hybrid systems in between. That research, based in part on data Oster collected during the pandemic, also found that in-person schooling was more common in districts that had higher test scores to begin with and that had fewer black and Hispanic students.
· A project called Education Recovery Scorecarda collaboration between researchers at Harvard and Stanford universities, looked at test scores from school districts in 29 states. It found that the average fully remote district lost more academic progress than others in the same state, which worked out personally, especially for math but also for reading.
Using this data, Nat Malkus, an education researcher at the American Enterprise Institute, divided school districts into three “buckets” based on how remote or in-person their students were. He calculated that students in the furthest group lost 60 percent of a school year in math, while those who spent the most time in classrooms lost 44 percent of a year.
For reading, the most remote group lost 33 percent of a year, versus 19 percent of a year for the most personal group.
“There is clearly a correlation between the duration of distance learning and student learning loss,” he said. But he added: “It’s also not as clean a relationship as everyone expected.”
That’s because there was huge variation across the country, with results in both remote and personal districts ranging widely. And there was one big outlier: California, where schools took a long time to bounce back, but academic performance wasn’t particularly bad compared to other states.
Sean Reardon, director of the Educational Opportunity Project at Stanford and a project leader on the Covid analysis, said Malkus’ calculations looked correct, but stressed that distance or in-person learning explained only part of the variation.
His team is working to see what other factors might explain the rest of the differences, such as local coronavirus rates or economic conditions. He speculated that parents’ financial problems, illness and social isolation all played a role.
“To reduce the educational effects of the pandemic to whether learning happened remotely or in person is to miss all the other ways the pandemic has disrupted the lives of children and parents and teachers,” he said. “There is a relationship, but it is not the only one.”
High poverty, steep decline
Not surprisingly, the students who already faced the greatest challenges suffered the greatest setbacks.
The Education Recovery data shows that students in the school districts with the highest poverty rates lost the equivalent of two-thirds of a grade in math, compared to the lowest poverty districts, which lost just under half a grade. The same was true for reading, although the distance was smaller. High-poverty districts lost 31 percent of a grade, compared to 25 percent in low-poverty schools.
The analysis of NWEA data showed that high-poverty schools were more likely to go out of business in the first place, and when they did, they suffered greater declines than low-poverty schools that did the same.
The report found that 30 percent of the difference in math achievement loss between high- and low-poverty schools was attributable to the increased likelihood that high-poverty schools were remote, and 50 percent was due to the impact of virtual learning.
“Distance education was a primary driver of widening the achievement gap,” the report found.
Several studies show that students are crawling out of the holes they fell into, although not all students and not as quickly as needed to achieve the expected academic growth prior to the pandemic.
ONE national survey using 2022 NWEA data found in the case of younger students, learning last year was close to pre-pandemic levels, helping students get started. But given the steep declines from the previous year, students were still far behind, especially in high-poverty schools.
The research also found that the rebound was stronger in math than in reading, which is important given that math took a bigger hit to begin with.
Also encouraging: renaissance, another testing company found that students last year grew academically at about the rate that would be expected in a pre-pandemic year.
But again, some subgroups of students grew faster than expected, including Asian American, Pacific Islander, and white students. Hispanic and especially black students grew more slowly than expected, as did students with disabilities.
“What worries me the most is the widening disparities we’ve seen,” said Karyn Lewis, director of the Center for School and Student Achievement at NWEA. “Everyone has been hurt, but some have been hurt more than most.”
Bigger kids, bigger problems
Several studies show that older students do not recover as quickly as younger ones. This trend is masked by much of the research because many of the state tests are administered only through eighth grade. But others include older students.
The Ohio data, for example, showed that students in grades three, four and six made up at least half of the lost ground in reading. Seventh-graders made some ground, but not much. There was little improvement in the eighth grade, and in the 10th grade the scores dropped again.
In mathematics there was modest progress in most classes, but in 10th there was virtually none.
That worries Kogan, the Ohio State researcher who did the analysis. “You’re talking about high schoolers with only a few years left,” he said. “We don’t have much time left to get them back on track. … The older students should be our top priority.”
That NWEA research from 2022 also found that younger students caught up much faster than older students.
The Renaissance data, which includes every class, showed the same. For reading, growth was about as expected or higher last year for students in fifth grade and younger, but lower than expected for everyone older than that.
The same pattern was true for math, where students in grades nine and up experienced slower growth than typical in the 2021-22 school year.
For these children, the downward spiral continues, said Gene Kerns, vice president and chief academic officer.
“The recovery actually plays out in very different ways for different kids,” he said. “The children in our elementary schools have done much better. It seems that the older the child, the more lingering the effects.”