Archiving Degree Audit Data to Measure and Reduce Lost Transfer Credit

Introduction

Since June 2019, the Articulation of Credit Transfer project (ACT) has focused on streamlining the information, advising, and administrative processes concerning how credits from one City University of New York (CUNY) institution transfer to another CUNY institution.^[1] This work addresses a critical challenge at CUNY, and indeed, across American higher education. When students transfer from one college to another, they frequently are unable to count their previously earned credits toward degree requirements at their new institution, jeopardizing their ability to earn that degree.

Research has shown that community college transfer students who have had most or all of their credits transferred are 2.5 times more likely to graduate compared to those who have had less than half of their credits transferred.^[2] Nationally, 43 percent of credits are wasted during transfer.^[3] At CUNY in particular, our data show that, prior to the ACT interventions, roughly 42 percent of students who transferred from Hostos Community College to Lehman College were unable to transfer all of their previously earned credits toward their degree at Lehman. On average, 18 percent of each of these students’ credits did not count towards degree requirements.

In fact, before the ACT project began, it was not possible to know, with accuracy, the number of credits that did not count toward a student’s degree following transfer. As explained in more detail below, this is because CUNY’s degree audit software, Ellucian Degree Works (“Degree Works”), overwrote each student’s degree audit data every time there was a change in the student’s record, leaving no information regarding whether, how, or when degree applicability changed.

This report describes one of the key innovations of the ACT project: instituting a daily archive of Degree Works data for students transferring between CUNY institutions participating in the project. Creating this archive has allowed the project team and other analysts to determine changes in how these students’ transferred courses counted toward their degree programs at their new institution. Access to this information permits three important actions:

• Aggregate-Level Analyses of Student Transfer Outcomes. The archive can be used to assess in the aggregate how many of students’ transferred credits count toward their degree program at their new institution. These results can be analyzed over time for positive or negative trends and for patterns within or across institutions to inform improvements in administrative policies and processes.
• Course-Level Analysis. The archive can be used to examine the ways that transfer affects the degree applicability of different courses under different circumstances, to inform improvements in academic and curricular decision making.
• Real-Time Monitoring of Transfer Students’ Fallthrough Courses. The archive can be used to monitor changes in an individual student’s degree audit when they transfer, allowing advisors to provide just-in-time support to help students make course and program decisions and changes that make the best use of their credits.

The remainder of the report contextualizes and explains the degree audit archiving effort, and how the ACT team has used the resulting data in these three ways.

Understanding “Fallthrough” and the Transfer Fallthrough Blindspot

We use the term “Fallthrough course” to describe a course on a student’s transcript that does not fulfill the general education, major, minor, or total-credits degree requirements of the program they are pursuing. There are two main reasons why a course for which a student earned credit at one CUNY institution may become a Fallthrough course upon transfer to another CUNY institution. First, a course may be “over the limit” if the receiving college places a limit on the number of transfer credits that can be applied to their degree or program requirements. Second, a course may be categorized as “overflow” if the student exceeds the number of “free electives” designated for their degree program.^[4] When a student has overflow credits, any additional elective credits that a student takes will also become overflow credits and will not count towards the student’s degree.

Credits that are categorized as “overflow” have the possibility of counting for major (or minor) requirements, or even for elective or general education credit, if a student changes their major or changes or adds a minor or second major. However, any credits over 60 for an associate’s degree or 120 for a bachelor’s degree, will always be in the overflow Fallthrough category (except for a small set of majors that require a greater number of total credits for a degree, or for students in a dual-degree program).

Currently, CUNY uses several different systems to maintain degree requirements needed to satisfy graduation credentials and to maintain a student’s academic record. A student’s academic record is maintained centrally in the university-wide student information system, CUNYfirst (CUNY Fully Integrated Resources and Services Tool). Program requirements are recorded in Degree Works, an enterprise platform managed centrally by CUNY but maintained as separate instances for each campus.^[5]

When a student transfers from one college to another, their academic record is processed using a set of CUNY-wide course transfer rules, which are available in CUNYfirst and accessible centrally, to determine how courses at the sending college translate into courses at the receiving college. Those transferred courses are then added to the student’s record at the receiving institution and reflected in the receiving college’s Degree Works instance, which assesses the courses against the relevant program requirements and identifies any new or continuing Fallthrough courses.

Staff on each campus can only see their own college’s Degree Works instance. In addition, as described previously, each Degree Works instance only presents the current audit for each student’s record—there is no log of prior audits or changes. Because of these Degree Works features, it has not been possible for anyone at CUNY—in a college or the central office—to see how courses entered or left the Fallthrough category upon transfer or as the result of student decisions about course registrations or about major and minor changes. Thus, there was no way to analyze which courses, departments, or schools generated the most Fallthrough credits and therefore might be problematic for transfer students. There was also no way of notifying students if one of their courses moved into Fallthrough due to a change in major or other cause.

For these reasons, understanding how many students are affected by Fallthrough courses, which courses are in Fallthrough, and the idiosyncrasies of Degree Works and its treatment of transfer rules, is crucial to improving credit transfer and graduation rates.

Degree Works Archiving for the ACT Project

To fill this information gap, one of the key initiatives of the ACT project was to begin daily archiving of Degree Works data for students transferring among the participating colleges. Daily archiving of Degree Works audit tables allows staff to observe, in real time, how transfer between institutions and receipt of an associate’s degree, as well as student decisions regarding course registration, major, and minor around the time of transfer, affect the types of courses and numbers of credits in Fallthrough. Daily archiving also creates a database of historical information with which to conduct aggregate analyses, such as determining how many credits are changing in their degree applicability for within-CUNY transfers, which categories of students are most likely to have courses in Fallthrough, and which colleges, majors, or disciplines have the most Fallthrough courses. The archive data can also be used to determine whether credit transfer policies are being followed.

Archiving began in September 2019 for students transferring from Hostos to Lehman, and in September 2020 for students transferring from Bronx Community College to Lehman. The other participating ACT colleges began archiving in January 2021. For each term since the first archive date, ACT has identified all students who are:

1. Direct transfers from an ACT community college to an ACT bachelor’s college. This does not include:
2. previous degree-seeking students at an ACT community college who have already earned a bachelor’s degree, and
3. non-degree students (i.e., College Now); and
4. Enrolled in an ACT bachelor’s college as of the term census date.^[6]

For each such student, the relevant Degree Works tables are copied and saved to an SQL server daily. Additional information on the sending and receiving institution, the total number of transfer credits, major and degree, and metadata on the classes in Fallthrough is also automatically added to the archive for each student. For a full list of variables in the archive database, please see Appendix A.

Interventions Using the Degree Works Archive

The Degree Works archive has many possible uses for research, policy, and practice related to transfer. As noted above, the ACT project has focused on three interventions: aggregate-level analysis of student transfer outcomes, course-level analysis, and real-time monitoring of transfer students’ Fallthrough courses.

Aggregate-Level Analyses of Student Transfer Outcomes

To assess patterns over time, we have defined cohorts of students who transferred from ACT community colleges to ACT bachelor’s colleges prior to the census date of each term since archiving began for that combination of institutions. Archiving began for Hostos-Lehman and BCC-Lehman transfers earlier than for transfers to Brooklyn College and Queens College. To take advantage of the analytical benefits of this longer time horizon for Lehman transfers, we present the Hostos-Lehman and BCC-Lehman cohort results separately, while aggregating the archive cohorts for the three main feeder community colleges to Brooklyn and the five main feeder community colleges to Queens.^[7]

Table 1 below shows the number of students in each cohort, by semester, since the beginning of archiving for each transfer pathway.

Over time, as shown in Figure 1, we observed a decline in the share of students with any course in Fallthrough. For example, the share of Hostos to Lehman transfer students with at least one transfer course in Fallthrough declined from 42 percent for the fall 2019 cohort to 28 percent for the fall 2021 cohort. Put another way, while only 58 percent of Hostos to Lehman transfer students in fall 2019 were able to count all their transfer credits toward their degree, 72 percent of similar students in fall 2021 were able to do so, a 24 percent improvement. A similar improvement is observed for BCC to Lehman transfer students over a shorter period: the share of students with at least one Fallthrough course declined from 29 percent for the fall 2020 cohort to 21 percent for the fall 2021 cohort. In other words, transfer students from BCC to Lehman who can now count their transfer credits towards their degree increased from 71 percent in fall 2020 to 79 percent in fall 2021, an 11 percent improvement.

Figure 1: Percentage of transfer students at Lehman College with at least one course in Fallthrough, by term and sending college

The percentage of transfer students from Hostos and BCC with at least one course in Fallthrough at Lehman. Source: Degree Works Archive

Yet, even while the share of transfer students who had Fallthrough courses declined, for those who did have at least one course in Fallthrough, the mean share of their courses in Fallthrough, slightly increased (Figure 2). For example, in fall 2019, among the 42 percent of students transferring from Hostos to Lehman who had at least one transferred course in Fallthrough at Lehman, the mean share of their courses in Fallthrough was 18 percent. In the fall 2021 cohort, only 28 percent of Hostos to Lehman transfer students had at least one transferred course in Fallthrough at Lehman, but the mean share of their courses in Fallthrough was 20 percent, an 11 percent increase. A similar trend is observed for transfer students from BCC to Lehman. The fall 2021 cohort of students with at least one transferred course in Fallthrough had a mean of 19 percent of their courses in Fallthrough, compared to 18 percent for analogous fall 2020 students.^[8]

Figure 2: Mean share of transferred courses in Fallthrough for transfer students at Lehman College with at least one course in Fallthrough, by Term and Sending College

Mean share of transferred courses in Fallthrough for students at Lehman College with at least one course in Fallthrough, by term and sending college. Only those transfer students with at least one course in Fallthrough are included in the analysis. Source: Degree Works Archive

Course-Level Analysis

In addition to the analysis of transfer student outcomes described above, the Degree Works archive can be used to identify Fallthrough patterns for particular courses and majors. One important application is to identify which courses taken at a community college most often end up in Fallthrough when students transfer to a four-year college. For example, for each semester archiving has been in effect, precalculus, introduction to sociology, and expository writing have been among the top 10 courses that end up in Fallthrough when students transfer from Hostos to Lehman.

Figure 3: Top 10 courses in Fallthrough at Lehman when transferred from Bronx Community College and Hostos*

*The y-axis illustrates the top 10 courses in Fallthrough at Lehman when transferred from Bronx Community College and Hostos. The number of times a course goes into Fallthrough is indicated to the right of the bars. Results are shown for the ACT Fall 2021 cohort. Source: Degree Works Archive

Figure 3 shows the top 10 courses in Fallthrough when transferred from BCC and Hostos to Lehman. Examining common Fallthrough courses in more detail can surface barriers transfer students face that might be alleviated through changes in policy or advising practices. For instance, the anatomy and physiology II course at Hostos often ends up as a Fallthrough course for students transferring to Lehman. Many of the students with anatomy and physiology II in Fallthrough were nursing majors at Hostos and were required to take that course. When they transferred to Lehman, however, many of these students did not pursue a standard nursing major, either because they decided they preferred a different major or because they were not accepted into the Lehman’s highly competitive nursing program. In that case, the anatomy and physiology II course from Hostos may not satisfy any major or general education requirements, and if the student has already exceeded their allotment of free electives and has satisfied the general education science requirements, this course ends up in Fallthrough.^[9] The frequency with which this situation occurs lends urgency to the need for additional seats in BSN programs as well as efficient off-ramps for those who aspire to, but ultimately do not enroll in them.

Understanding the nuances in how Degree Works treats certain transfer rules are concepts that are also important for practitioners to understand. For example, expository writing is a Pathways general education course and should always transfer as such.^[10] However, when a student holds an associate’s degree at the time of transfer, Degree Works treats the general education requirement as met and therefore arbitrarily assigns a subset of transferred general education courses to Fallthrough. This means that a course such as expository writing can show up in the archive as frequently being in Fallthrough when there has been no problem with this course transferring.

Fundamentals of interpersonal communication is yet another course at BCC that undergoes the same treatment described above. This course is often taken to satisfy the individual and society general education requirement for Pathways, but for students transferring with an associate’s degree, Degree Works treats the general education requirement as fulfilled and the course ultimately ends up in Fallthrough.

In looking at the results of the archive Fallthrough analyses, it is important to distinguish between general education courses that were arbitrarily placed into Fallthrough by Degree Works and courses in Fallthrough as a result of not being able to transfer.

Real-Time Monitoring of Transfer Students’ Fallthrough Courses

In addition to the aggregate analyses described above, archiving Degree Works data has enabled advisors to monitor the records of new transfer students and identify changes in course Fallthrough status in real time. Using the archive, advisors can observe a new transfer student registering for a Fallthrough course, prompting advisors to contact the student to suggest adding a minor, changing a major, swapping a course, or requesting a substitution to reduce the courses in Fallthrough. This results in more of the student’s transferred and current registrations counting toward their degree program requirements, saving the student time and money. Since spring 2020, this model of advising at Lehman has resulted in 600 transfer applicants making changes to their course schedule based on recommendations from their advisors, and these changes have increased the number of credits students can count toward their degrees compared to their initial audit. Similarly, at Brooklyn College after reviewing 132 student files, 36 students were contacted about making changes to their course schedules, leading to an increase in the number of credits counted towards those students’ degrees.

This archive-enabled advising strategy can mean the difference for some students between receiving and not receiving state financial aid. To be eligible for the New York Tuition Assistance Program (TAP) financial aid benefits, a student must be enrolled full time and take 12 or more credits per semester that are applicable to their degree requirements. If students’ transferred courses cause any of their current term courses to be in Fallthrough (and therefore not applicable to their degree), these students could end up with fewer than the 12 degree-applicable credits necessary for TAP eligibility—losing that aid. Using the archive to monitor and address Fallthrough can help students reach the 12-credit threshold. Because of outreach by Lehman’s Fallthrough auditor from May through August 2021, 19 students in the fall 2021 cohort, whose initial registrations left them ineligible for TAP, were able to make registration changes that made them eligible, receiving an average award of nearly $1,489. These endeavors have gone beyond helping students in the ACT cohort, with an additional eight students from non-ACT cohorts assisted to earn a mean of $1,954 per student in TAP funds that they otherwise would have received.

While Lehman pioneered this advising strategy using a highly manual process, in fall 2021 the ACT team created an “Overflow Dashboard”: an automatically updated list of students with courses newly in Fallthrough. In addition to allowing advisors to monitor Fallthrough changes more easily, the dashboard has a feature that allows advisors to automatically generate emails to students on the list alerting the students to any new Fallthrough credits in their records and suggesting a remedy. The dashboard also includes the functionality to log actions taken by the advisor or the student. Already in use across the seven ACT colleges, the dashboard can be used by advisors at any CUNY college.

Moving Forward

Archiving of Degree Works has proven to be a powerful tool for college administrators and advisors who work with transfer students day after day. The ability to observe high-level trends or even more granular data, such as specific Fallthrough courses, has allowed advisors to implement strategic interventions to address specific problems. It is evident that intervention efforts and the work of archiving is paying off. The data given above show us a general trend up in the number of students who became eligible for TAP and a year-by-year decline in the percentage of students who have Fallthrough courses.

The support many students are now receiving due to more targeted advising can decrease the time, money, and energy students spend having to take additional courses to satisfy degree requirements previously satisfied by credits that were lost. Repeating courses can also lead to a decrease in morale and motivation, consequences that will increase any difficulties these students have in graduating and entering the workforce.

The analyses laid out above are critical for tracking the progress of the ACT project; however these analyses represent only a fraction of the useful information in the archive. With additional development and use, the archive can serve as a basis for even more analyses and applications to improve the transfer process at CUNY and beyond.

Appendix A

Variables in the Degree Works Archive

• Sending and receiving institution
• Emplid (CUNY’s unique student id)
• Total transfer credits
• Term student was admitted to Lehman
• Information on students’ majors, minors, and concentrations
• Six “change flags” indicating the date of a significant change in a student’s academic record:
  • Transfer credit change
  • Academic plan (major or minor) change
  • Course registration change (add, drop, swap)
  • Scribe change (degree requirements coding change in Degree Works)
  • Exception change (an exception to a degree requirement was added or removed)
  • Overall CUNYfirst change (this could be student’s name, student’s group, administrative hold, etc.)
• The sending course, receiving course, subject, catalog number, course title, and course ID
• Course grades and credits
• Pathways general education course area (if any)
• Students with a pending associate’s degree
• Students’ prior degrees
• Over the limit credits and classes
• Overflow credits and classes
• Date when archive was updated

Endnotes

1. ACT is a collaboration among Ithaka S+R, CUNY Graduate Center researcher Alexandra Logue, the CUNY Office of Applied Research, Evaluation, and Data Analytics (OAREDA), and seven CUNY colleges: Bronx Community College (BCC), Brooklyn College, Guttman Community College, Hostos Community College, Lehman College, Queens College, and Queensborough Community College. ACT is funded by grants from the Heckscher Foundation for Children and the Carroll and Milton Petrie Foundation, as well as by CUNY. For more information on the origins and development of ACT, see Martin Kurzweil, Cindy Le, and Alexandra W. Logue, “Ithaka S+R to Expand Transfer Improvement Efforts with CUNY,” Ithaka S+R, 15 December 2020, https://sr.ithaka.org/blog/ithaka-sr-to-expand-transfer-improvement-efforts-with-cuny/. ↑
2. David B. Monaghan and Paul Attewell, “The Community College Route to the Bachelor’s Degree,” Educational Evaluation and Policy Analysis 37, no. 1, March 2015: 70–91, https://doi.org/10.3102/0162373714521865. ↑
3. “Higher Education: Students Need More Information to Help Reduce Challenges in Transferring College Credits,” US Government Accountability Office, 14 August 2017, https://www.gao.gov/products/gao-17-574. ↑
4. “Free electives” are courses that can be applied to reach the 120 credits required to graduate and are in addition to the credits that satisfy the general education, major, and any minor requirements of the degree program. At transfer, some students are left with more course credits than can be applied to degree program requirements, including free elective requirements, and those excess credits are then categorized as “overflow.” ↑
5. “Instance” in this context refers to each college’s own copy of the Degree Works database. ↑
6. Census date refers to the day immediately following the term’s final course add deadline. ↑
7. Brooklyn College’s main feeder schools are Bronx Community College, Hostos Community College, Kingsborough Community College, Guttman Community College, and Queensborough Community College. Queens College’s feeder schools are Bronx Community College, Hostos Community College, and Queensborough Community College. ↑

8. With only one spring cohort and one fall cohort in the archive for transfers to Brooklyn College and Queens College, it is not yet possible to assess trends in Fallthrough outcomes. In addition to the relatively short assessment period, spring and fall transfer students differ in systematic ways that make direct comparisons between spring and fall unreliable. ↑

9. This can happen even for students who choose to pursue Lehman’s online nursing major. Because this online major is for students who already have an RN, there are no degree requirements that the anatomy and physiology II course can satisfy. ↑
10. Pathways is a system of general education requirements and transfer guidelines across CUNY. Credit from common core courses and gateway courses for popular majors will transfer if students transfer to another CUNY college. ↑