Can changes to the advising process help less advantaged students persist and graduate? This is the question at the heart of an $8.9 million First in the World validation grant awarded to Georgia State University, on behalf of the University Innovation Alliance (UIA).  The Monitoring Advising Analytics to Promote Success (MAAPS) project seeks to bring to scale and test the impacts of a proactive advising system for low-income and first-generation students. Ithaka S+R was brought on to the project as the independent evaluator of the 10,499-student randomized trial, and we are now sharing the results from the first year of the study.

The MAAPS project was officially launched at eleven large public research universities that form the UIA during the fall 2016 term with more than 10,000 first-time, first-year students, and is presently in its second year of implementation. Students in the treatment group are offered intensive and proactive outreach, degree planning activities, and targeted interventions from dedicated MAAPS advisors, in addition to business-as-usual advisement at the institution. Students in the control group receive business-as-usual advisement at their institution and do not have access to MAAPS advisors. We examined the impact of assignment to MAAPS advisement on student outcomes in the first year of the intervention, as well as variations in the implementation of the intervention across the participating sites, and present our analyses and findings in this early impact findings report.

The results indicate that, after one academic year, MAAPS did not have a significant impact on students’ academic achievement and persistence outcomes across the sites in the aggregate. However, at Georgia State, students in the MAAPS program did see some better outcomes than students in the control group—on average, they accumulated 1.20 more credits, passed 3 percentage points more of the credits they attempted, and achieved GPAs that were 0.17 points higher. Additional analyses indicate that it is the lower-performing Georgia State students in the MAAPS program, in particular, that are attempting and earning more credits and achieving higher GPAs than their counterparts in the control group.

Because the anticipated impacts of MAAPS advisement are longer term, the first-year findings are not surprising. Measurable early college outcomes pertaining to progress and achievement begin to manifest themselves more clearly during students’ sophomore year, and MAAPS advisement may be especially impactful after students’ first year, when typical wrap-around institutional supports for new students tend to diminish. [1] As such, any average impacts of MAAPS on student outcomes are difficult to detect early in the intervention. Additionally, a number of sites faced some implementation challenges in the first year of the study, causing them to offer key components of MAAPS with less intensity and frequency than originally planned. At Georgia State, where MAAPS was conceived, the program was implemented with a high degree of fidelity in the first year. Georgia State was able to rely on existing centralized advisement structure, established culture and practices surrounding the use of in-time student data for proactive advisement, and precursors to MAAPS degree mapping in the form of paper-and-pen first year maps developed by advisors as part of business-as-usual advisement.

While we found no observable impact on student persistence and academic achievement at the other sites, there were some promising signs. The results of a first-year student advising survey and focus groups with MAAPS students at eight participating sites suggest that treatment group students have more contact with their advisors and higher levels of institutional know-how than students in the control group.  Such experiences and know-how may contribute to longer-term positive impacts on achievement and progress. Combined with the positive early impacts of MAAPS advising at Georgia State, these results hold promise for the broader impacts of MAAPS advisement in the second and third years of the intervention.

[1] Mary Stuart Hunter et al, Helping Sophomores Succeed: Understanding and Improving the Second Year Experience, Jossey-Bass (2010).