Note: This blog reflects updates to an earlier version published on September 4, 2020 that described results from preliminary analyses of the first group of 12 institutions. The updated results include 18 institutions, total, and also reflect a minor change in the methodology used to predict scores across all institutions. Both the increase in the number of schools included in the analysis and the methodology change are responsible for changes in the results. The biggest change is that the graduation rate outcomes of Pell recipients and students from underrepresented minority groups are better than previously reported.

In May 2020, Ithaka S+R partnered with the Council of Independent Colleges (CIC) and a group of its member institutions to assess the impact of the COVID-19 pandemic’s disruption to the Spring 2020 term on students’ academic outcomes, and to inform subsequent institutional decision-making. We concluded our work with nineteen institutions across two groups this month, and each participating institution received a customized confidential report of findings.

In this blog post, we describe the project and our methodology, and present high-level findings from the participating institutions. We also discuss our plans for similar research that aims to assist institutions in tracking student outcomes as they adjust course during the ongoing pandemic.

For this project, we collected and analyzed de-identified student demographic and outcomes data of degree-seeking undergraduates enrolled during the spring terms of 2016 through 2020. We estimated how students’ outcomes in the spring of 2020 compare to their otherwise anticipated outcomes in the absence of the pandemic. To do this, we regressed student outcome data during the spring terms of 2016 to 2019 on students’ demographic characteristics. The resulting constant and coefficient estimates were used to predict the outcomes of students enrolled in the Spring 2020 term given their student demographic characteristics. We also benchmarked each institution’s results against a set of self-selected peer institutions to allow institutions to further contextualize the results and assign meaning to them. All analyses were conducted across each institution’s full sample, as well as institutional subsamples to uncover potential disparate outcomes among particular student groups (e.g. Pell recipients and students from underrepresented minority groups). For more information on the student outcomes we analyzed and how the findings are presented, you can download a sample report. 

The results described below reflect analyses across the 18 bachelor’s degree-granting institutions in our sample (one of the 19 institutions is an associate degree-granting institution so we excluded them from this analysis). A few high-level findings comparing students’ actual outcomes to their predicted outcomes in the absence of the pandemic stand out. It’s worth noting that the differences are not large and have not been tested for statistical significance, and like all short-term findings, it’s unclear at this time whether they will persist beyond the Spring 2020 term.

  1. Students’ Spring 2020 term GPA was higher than predicted.

Similar to what was reported in the initial blog, across the entire sample, students’ Spring 2020 term GPA was 0.14 GPA points higher than predicted, likely driven by a few factors. In addition to passing a higher share of the credits that they attempted, students passed nearly half a credit more using the pass/fail option than predicted. Students typically use this option instead of a letter grade when they anticipate earning a relatively low grade in a particular course – as courses completed with a “pass” do not affect students’ GPA. Anecdotal evidence suggests that institutions gave students greater flexibility and time to use this option. Though it may have had a positive impact on GPA, it’s unclear what other impacts it may have on students’ long-term academic outcomes, such as enrolling in upper-level courses or applying to graduate schools that require students pass prerequisite courses with letter grades. An alternative explanation for the higher-than-predicted GPAs is that faculty graded students more leniently during what they may have perceived to be unfair or inadequate conditions for evaluating students. These findings were consistent across student subgroups as well.

  1. Students experienced higher-than-predicted graduation rates.

Across the entire sample, the share of fourth-year standing students (typically defined at institutions on the semester system as students who have earned at least 90 cumulative credits) who graduated at the conclusion of the Spring 2020 term was more than 1.6 percentage points higher than predicted. In contrast to what was reported in the initial blog, student subgroups had similar experiences. In fact, the graduation rate of fourth-year standing Pell recipients and underrepresented minority students were each 1.8 percentage points higher than predicted; among female students, it was 1.3 percentage points higher than predicted. With that said, it’s worth noting that the actual graduation rates of Pell recipients and underrepresented minority students were more than two and six percentage points lower than the actual graduation rates of all students, respectively. These findings are consistent with well-known and long-standing racial and socioeconomic completion gaps at colleges and universities in the United States.

Helping institutions understand the impact of the pandemic on students’ outcomes is all the more important considering that many schools are offering most, if not all, Fall 2020 courses online as well. Although the Spring 2020 term is much more accurately described as emergency remote instruction than online learning per se, the Fall 2020 term retains emergency elements and uncertainty as the pandemic continues to unfold. Understanding impacts, and especially disparate impacts, during the Spring 2020 term can help institutions target resources and supports to the most affected students. 

To help institutions understand whether the plans they developed and implemented over the summer were effective, and how their students are continuing to fare as the consequences of the COVID-19 pandemic continue to unfold, Ithaka S+R is developing a similar project to track and benchmark student outcomes on a continuous basis over the next several academic years. It will build off the lessons learned in the current project and include additional features and supports so institutions can make strategic use of project findings.

Ithaka S+R is seeking additional partnerships along with CIC’s continued involvement. If your institution is part of a broader group, network, or consortium of institutions that may be interested in participating, we invite you to contact us through the comments section below or at daniel.rossman@ithaka.org to learn more.