Higher Ed Insights: Results of the Fall 2015 Survey

In fall 2015, Ithaka S+R invited a select group of higher education administrators and experts to join a panel of advisors. One activity of the panel, which consists of 110 members with diverse backgrounds and perspectives, is to take part in semi-annual surveys on issues of national importance in higher education. Ithaka S+R will analyze and publish the results of these surveys to inform the broader higher education community about the panel’s views on current debates, initiatives, and challenges. The results of the Higher Ed Insights surveys will also help guide Ithaka S+R’s research agenda.

This report presents findings from the first panel survey, administered between November 16^th and December 27^th, 2015. The survey focused on innovative initiatives and strategies aimed at improving three student-centered outcomes: degree completion rates, the quality of student learning, and affordability (i.e., students’ ability to cover the costs of earning a degree).^[1] It also solicited panel members’ opinions on the current state of undergraduate education in the US and obstacles to successful innovation.

The narrative section of the report describes Ithaka S+R’s analysis of the survey results. Aggregate responses to all close-ended survey items are provided in the Appendix.

Summary of Findings

The 96 panel members who responded to the survey come from a broad range of organizational contexts. Yet for the most part, they agreed on the status of American higher education, its prospects for the future, the promise of various initiatives, and obstacles that stand in the way of improvement.

Respondents rate the current state of higher education in the US at just above neutral, but most are hopeful that the sector will improve in the next ten years. In their view, improvements in degree completion, the quality of student learning, and affordability are important—and roughly equally important—to achieve sectoral improvement.

To improve degree completion, respondents see the most potential in guided pathways and proactive advising strategies—efforts to provide students with more structured postsecondary experiences, with more support. These results were particularly popular among respondents at public colleges and universities.

Respondents view intelligent adaptive learning technologies—instructional software that adjusts the material presented to students in real time, as students interact with the software—as the most promising initiative for improving the quality of student learning. Systematic assessment of student learning is also viewed favorably by a majority of respondents as an intervention to improve student learning.

Asked to rank a list of efforts to make earning a college degree more affordable for students, respondents were most likely to select as their top choice unbundling college credits and services—allowing students to pay for only those courses and services they want, and to combine experiences from multiple institutions (or other providers) to earn a degree. The preference for unbundling in the affordability context presents an interesting contrast to the preference for guided pathways in the degree completion context: the former works by increasing the range of choices available to students, while the latter works by limiting student choices. This tension—between the strategies and possibly between the goals of degree completion and affordability—merits deeper scrutiny.

Universal free tuition for two years, an initiative advanced by President Obama and much discussed in the 2016 presidential primaries, drew the most varied support from our respondents. We asked about universal free tuition in the context of both degree completion and affordability, and in each case responses were distinctly varied—a sharp contrast to most of the other initiatives, about which respondents generally agreed. It seems that our expert respondents’ opinions about this issue are as diverse as those of the broader higher education community.

When asked about obstacles to successful innovation in American higher education, respondents most frequently cited barriers grounded in institutional culture and structures. Although they recognize the role played by ineffective public policy, low and declining public resources, and market inefficiencies, our survey respondents, mostly higher education insiders, see the biggest roadblocks to innovation inside the academy. These include misaligned incentives for faculty, lack of clear vision by leaders, and commitment to outdated instructional and organizational models.

Respondents from the Higher Ed Insights panel see great potential in a number of current and emerging initiatives to improve degree completion, student learning, and affordability. It is not, according to respondents, a lack of good ideas that holds the higher education sector back. Rather, to improve student prospects, leaders and faculty will have to overcome sometimes deeply entrenched cultural norms and organizational structures to put those ideas to work. That challenge is not small, but our respondents are optimistic that it will be met.

Detailed Survey Findings

Panel and Respondents

The Higher Ed Insights panel consists of 110 higher education experts, who were invited to join based on their expertise in the field and their affiliation with innovative or influential institutions. A total of 96 panel members completed the fall 2015 survey (87% of the panel).

Respondents’ Affiliations

The majority of respondents are primarily affiliated with an institution of higher education (68%). Of those, most respondents are affiliated with public institutions: 35 percent are affiliated with public four-year universities and colleges, 12 percent with community colleges, and 9 percent with public college or university systems. Another large group of respondents from higher education institutions are affiliated with private non-profit four-year universities and colleges (40%). Three percent of higher education institutions represented are for-profit colleges. The remaining third of respondents are primarily affiliated with other types of institutions, such as research institutes or think tanks, member associations, and philanthropic foundations.

Figure 1.

Respondents primarily affiliated with higher education institutions (n=65) represent 54 institutions that differ widely in size and student composition. For instance the size of their undergraduate student population ranges from less than 700 to over 150,000, with an average of approximately 20,000 students (Median=17,620). Similarly, the percentage of Pell-eligible students enrolled at these institutions ranges from 11 to 75 percent, averaging 29 percent (Median=22%).

Respondents’ Roles

Many respondents identified their roles at their primary institution of affiliation as chief executives, including Chief Executive Officers, Presidents, or Chancellors (35%). Additionally, eight percent held positions as Provosts, Chief Academic Officers, or Vice Presidents. Other reported roles include other administrative positions (28%), such as chief innovation officer, researchers (22%), faculty members (21%), and policymakers, advocates, program officers, or consultants (13%).

Respondents’ Experience

Respondents were asked to report the number of years they have been professionally involved in higher education, including all experiences that have contributed to their current expertise in the field. On average respondents reported 28 years of experience in higher education, ranging from three to 52 years.

Figure 2.

As would be expected, respondents who reported holding administrative positions at higher education institutions have significantly more years of experience in the field (M=30.29; SD=1.16) than respondents who held other positions at such institutions or elsewhere (M=24.89; SD=1.92).

Finally, most respondents (80%) reported having taught at least one undergraduate college or university course in the past, as the primary instructor.

Current State of Undergraduate Education in the US

How Would You Rate the Current State of Undergraduate Education in the US?

Respondents were asked to rate the current state of undergraduate education in the US, and their hopes for its future. On a scale ranging from “extremely poor” to “excellent,” respondents rated the current state of undergraduate education in the US as slightly above neutral. Close to a quarter (23%) rated it as below neutral while 16 percent chose one of the top two ratings.^[2]

Figure 3.

How Hopeful Are You That the State of Undergraduate Education in the US Will Improve Significantly by 2025?

Respondents who rated the current state of undergraduate education in the US as less than “6” (n=79) were asked how hopeful they are that it will improve significantly by 2025. On average these respondents gave ratings above neutral, and more than a third gave one of the top two ratings (35%).

Figure 4.

Responses to these two questions did not differ significantly based on respondents’ institutional affiliation or role.

How Urgent Is It to Address Each of the Following in Order to Improve Undergraduate Education in the US?

Finally, respondents consider it very urgent to address degree completion rates, the quality of student learning, and affordability in order to improve undergraduate education in the US. With 1 = “Not at all urgent” and 7 = “Extremely urgent”, 85 percent of respondents selected a rating above 4 (neutral) for each of these three issues.

Figure 5.

It is noteworthy that on average, responses were similar across these three issues, with no significant subgroup differences. Interestingly, however, ratings for the urgency of addressing degree completion rates were positively correlated with respondents’ years of experience in the field (r=.37; p<.001).

A closer look reveals that the few respondents who consider addressing degree completion rates as non-urgent (neutral or less) were almost exclusively researchers and faculty members (9 out of 10) with fewer years of experience in the field than all respondents on average (M=17.40; SD=7.75). Conversely, respondents who consider it extremely urgent held mostly administrative or senior management positions (24 out of 28), and consequently also have more years of experience in the field (M=29.68; SD=10.75).

This finding may, in part, reflect the increasing external pressures placed on administrators to improve degree completion rates at higher education institutions; pressures that researchers and faculty members may not feel as acutely. Those who rated degree completion as less urgent may also have other priorities: one panel member who identified as a researcher commented that the focus on completion is misaligned to the public’s interest in the real costs and economic value of college education.

Degree Completion Rates

Respondents were presented with the following five initiatives/strategies, and asked to rate how promising each initiative is for improving undergraduate degree completion rates.

Guided pathways is an approach that presents college courses in the context of highly structured, educationally coherent program maps that align with students’ goals for careers and further education, including alignment with articulated transfer pathways between colleges and universities.^[3]

Proactive Data-Informed Advising draws on predictive analytics and new technologies to offer students intensive proactive advisement that helps them establish individualized academic plans, navigate key academic choices, and receive timely targeted advising interventions when they go off-path (e.g. Integrated Planning and Advising Services).^[4]

Universal Free Two-Year College Education (e.g. America’s College Promise) would offer first-time community college students a full tuition waiver for two years, contingent upon satisfactory academic progress and enrollment in programs that lead to transfers or in-demand jobs.

Credit for Off-Campus Coursework entails expanding opportunities for students to earn college credits by completing coursework through a variety of other institutions, including third-party online courses (e.g. MOOCs), and pre-college programs (e.g. dual-enrollment, AP, international baccalaureate).^[5]

Performance Pay ties institutional funding or faculty compensation structures to student outcome measures and progress toward target student outcome goals.^[6]

How Promising is Each Initiative for Improving Undergraduate Degree Completion Rates?

On average, respondents rated proactive data-informed advising and guided pathways as the most promising for improving undergraduate completion rates, and universal free two-year college education and performance pay as the least promising (1 = “Not at all promising”; 7 = “Extremely promising”).

Figure 6.

Proactive data-informed advising and guided pathways were given one of the top two ratings by 72 percent and 66 percent of respondents, respectively. These initiatives were positively correlated with each other (r=.41; p<.001), and received significantly higher ratings than the other three listed initiatives (p<.05).

Although all respondents affiliated with institutions of higher education rated these strategies highly, those affiliated with public institutions rated both strategies as more promising than respondents affiliated with private non-profit institutions rated them (p<.05). This may be because these initiatives are especially useful for intervening with large groups of students and at-risk students, both of which are more common at public institutions.

Figure 7.

Although universal free two-year college education and performance pay were rated as significantly less promising for improving undergraduate degree completion rates than other initiatives, they generated the greatest variation in responses across the board.

Figure 8.

Larger variation in ratings for the promise of universal free two-year college education may be a result of its relative novelty as a potential nation-wide initiative or the strong political valence of the strategy. There were no notable subgroup differences for this question.

Large variation in ratings for the promise of performance pay may be in part due to subgroup differences in relation to this initiative. More specifically, individuals affiliated with higher education institutions rated performance pay as less promising (neutral) than individuals affiliated with other types of institutions (somewhat promising; p<.05).

Figure 9.

Variation may also be due to how the initiative was defined in the present survey: tying either institutional funding or faculty compensation structures to student outcomes, rather than focusing on one or the other. Open-ended survey responses suggest that many respondents see promise in changes to faculty compensation structures at large, including in some cases changes that incorporate faculty performance pay, while few discussed performance pay at the institutional level.

Quality of Student Learning

Respondents were presented with the following five initiatives/strategies, and asked to rate how promising each initiative is for improving the quality of undergraduate student learning.

Technology-Focused Faculty Development Incentives reward faculty members financially and professionally for participating in programs or innovations that help them achieve learning goals in their courses using new technology tools.^[7]

Intelligent Adaptive Learning Technologies are computer-based and/or online educational systems that modify the presentation of learning material in response to real-time student performance and interactions.^[8]

Competency-Based Education allows students to advance through courses and degree programs at their own pace, rather than focus on seat time or credit hours, based on their ability to demonstrate different “competencies” deemed necessary for academic and industry success in their field.

Digital Game-Based Learning incorporates educational content or learning principles into online or video games with the goal of improving student engagement, content knowledge, and processing skills. Its applications draw on the constructivist theory of education.

Systematic Assessments of Student Learning involve yearly assessments and reporting of concrete student learning outcomes based on systematic predetermined programmatic goals and guidelines, using direct evidence of student learning.^[9]

How Promising is Each Initiative for Improving the Quality of Undergraduate Student Learning?

Respondents rated intelligent adaptive learning technologies as the most promising strategy. Fifty-six percent of respondents gave adaptive learning technologies one of the top two ratings, and it received significantly higher ratings than the other initiatives (p<.01). On the other hand, digital game-based learning was rated as the least promising (p<.01).

Figure 10.

The other three initiatives listed—technology-focused faculty development initiatives, competency-based education, and systematic assessments of student learning—received average ratings above neutral for improving the quality of student learning. Although average ratings for these initiatives did not differ significantly, the systematic assessment of student learning was more often given one of the top two ratings (52%) than the two other initiatives (30% and 33%).

Figure 11.

Affordability

Respondents were asked to rank the following five initiatives that aim to improve affordability based on the extent to which they support, or would support, their adoption, keeping in mind other potential effects of these initiatives (1=Support the most; 5=Support the least).

Unbundling college credits and services e.g. Students only pay for specific courses & services of their choosing, across different institutions of their choice.

Income-based loan repayment plans e.g. Monthly repayments of 10-15% of the graduate’s discretionary income.

Risk-sharing Institutions share student loan risks (e.g. face penalties for poor completion or student inability to repay debt).

Universal free college education for two years e.g. America’s College Promise.

Passing institutional savings onto students e.g. Institutions use savings from cost-cutting initiatives to offer lower or freeze tuition rates.

Ranking of Affordability Initiatives

Unbundling college credits and services received the most support of the five affordability initiatives. It was most frequently ranked at the top, with 59 percent of respondents ranking it first or second, and the least frequently ranked at the bottom. It seems contradictory that respondents support unbundling as an affordability measure and also find much promise in guided pathways as a degree completion measure – the former increases students’ range of choices while the latter limits it. It may be that respondents perceive some tension between the two goals, or that they envision unbundling taking place within more limited structures. It may also be that respondents had different types of students or institutions in mind when considering affordability and degree completion.

Figure 12.

Institutional risk-sharing was the second-most-popular affordability initiative, ranked first or second by 45 percent of respondents. On the other hand, respondents were least likely to support income-based loan repayment plans as a strategy for improving affordability. Low support for this initiative, coupled with relatively high support for risk-sharing, suggests that respondents favor financing initiatives that place some responsibility on institutions for the outcomes of their students, rather than leaving taxpayers to bear the risk of student default.

As it did in relation to degree completion, universal free two-year college education elicited a wide range of views in relation to its role in improving affordability, with 23 percent of respondents ranking it at the top and 29 percent ranking it at the bottom. As one would expect, respondents affiliated with public higher education institutions were more likely to rank this initiative in the top two (49%) than those affiliated with private institutions (21%), although the difference in the average ranking across these two groups was not statistically significant (p>.10). Respondents affiliated with community colleges (n=8) showed similarly varying levels of support for this initiative, with half ranking it in the top two and half in the bottom two.

Finally, passing institutional savings onto students received low levels of support on average, especially from higher education administrators who gave this initiative a significantly worse average ranking than respondents who held other positions (p<.05).

Obstacles to Successful Innovation on US Campuses

What are the most significant obstacles to successful innovation on US college campuses?

Respondents were asked to list up to three obstacles to successful innovation on US college campuses, in relation to undergraduate completion, the quality of student learning, and/or affordability.

All respondents listed at least two obstacles, most listed three, and some listed more. As a group, respondents listed 267 obstacles to successful innovation, or to the success of higher education in general, that included enough description for adequate coding. These obstacles were categorized based on six common themes:^[10]

• Institutional Culture and Structures (46%)
• Funding and Resources (18%)
• The Higher Education Market (11%)
• Governmental Policies (10%)
• Student Characteristics (8%)
• Data and Evidence (6%)
• Other uncategorized (4%)

The next sections discuss each of these obstacle types in more detail and offer a selection of direct quotes from respondents.

Institutional Culture and Structures as Obstacles

Close to half of all the obstacles identified by respondents pertained to the culture and structures of higher education institutions, with 80 percent of respondents describing at least one such obstacle. Respondents listed institutions’ visions and goals, hiring policies, models of course delivery, and leadership priorities and styles as significant obstacles to successful innovation on campuses or to the success of higher education more generally. Discussions of institutional culture centered mostly, though not exclusively, on faculty resistance to change or disinterest in student-centered outcomes, the problematic nature of faculty rewards and incentives, and issues of governance.

Respondents with different roles and affiliations were equally likely to list obstacles pertaining to institutional structures and culture. Differences pertaining to the specific types of institutional obstacles discussed (e.g., faculty incentive structures) were not analyzed.

“Innovation is dependent on the creation of institutional cultures that [are] able to support innovative practice – culture building is a challenging enterprise and is generally overlooked as a goal.” Administrator, community college

“A one-size-fits-all model of course delivery in which time is constant and learning variable. Large class sizes, and lack of individual attention.” Administrator, public research university

“A 20th century vision of the job description of faculty.” President, non-profit organization

“Four years of residential education just after high school is a very high-cost delivery mechanism and virtually no one is calling for its abolition or a fundamental rethinking. This may well be because [it] is baked into the business model of the vast middle as well as the elite top of the sector. There’s a LOT of institutional inertia on this.” Faculty member, private non-profit research university

“Significant innovation requires a willingness to change on the part of all stakeholders. This seems like an intractable hurdle at most institutions where forms of shared governance appear increasingly anachronistic and disconnected from daily life.” Senior staff member, foundation

“Misalignment of faculty interests/incentives and larger institutional goals and societal goals.” Administrator, private non-profit research university

Resource and Funding Obstacles

The second most common type of obstacle to innovation described by respondents are issues of resources and funding, raised by 45 percent of respondents.

Respondents point mainly to inadequate funding from states and the federal government in general or for specific types of programs, as well as the problematic allocation of funds and resources within institutions. Some respondents also described a paucity of human, technological, and material resources within institutions, as well as the rising costs of providing higher education.

As a group, higher education administrators were significantly more likely to list at least one obstacle pertaining to resources and funding than other respondents (56% vs. 29%; p<.01). Respondents associated with public higher education institutions were not significantly more likely to discuss resource- or funding-related obstacles than their counterparts at private non-profit institutions (54% vs. 46%; p>.10).

“Under-investment in financial support for low income and middle income students – such that students spend too much time working outside of their academic programs, accumulate too much debt, and narrow their academic horizons.” Chancellor, public research university

“State cuts in funding for higher education.” President, membership association

“Too much emphasis on college amenities to [the] neglect of instructional quality and rigor. Assumption that costs can continue to routinely increase.” Senior administrator, community college

“Lack of resources on campuses serving most at-risk students.” Researcher, research/policy institute or think tank

“High cost of human-intensive education.” Administrator, private liberal arts college

“Decrease in state support, changes in financial aid at the federal level while real costs go up, putting schools in a bind and needing to increase charges […] With limited budgets and increasing demands, we do not have resources to invest in innovation.” Senior administrator, public college or university system

“Lack of resources in public universities and community colleges.” Faculty member, public research university

The Higher Education Market

About a quarter of respondents (27%) described obstacles pertaining to higher education market forces, including the values and actions of the American public and media, and the role of market competition (or a lack thereof). Respondents affiliated with higher education institutions were slightly, although not significantly, less likely to discuss such an obstacle than respondents affiliated with other types of institutions (23% vs. 35%; p>.10).

“Lack of competition on price and value.”  Researcher, research/policy institute or think tank

“Higher education exists in an imperfect market making it less responsive than would otherwise be expected from the intense political, financial, and consumer pressures bearing down upon it.” Senior staff member, foundation

“Distorted perceptions of the value of higher education and continued negative press.” Faculty member & administrator, public research university

“US News & World Report rankings.” Researcher, research/policy institute or think tank

Governmental Policies

A quarter of respondents (25%) described obstacles pertaining to state and federal policies, most often citing inappropriate or missing incentive structures as well as problematic regulations and accreditation requirements. Respondents associated with private higher education institutions were somewhat, although not significantly, more likely to discuss such obstacles than their counterparts at public institutions (31% vs. 16%; p>.10).

“[Lack of a] federally mandated and state-run information system that ties secondary and postsecondary transcript data, wages and occupations to programs of study.” Researcher, private non-profit research university

“The accreditation system prevents competition.” President, private non-profit research university

“Current accreditation processes hamstring innovation without ensuring quality […]” Researcher, research/policy institute or think tank

“The lack of financial incentive for institutions to innovate and put into place effective practices.” President, membership association

Student Characteristics

A relatively small number of respondents (21%) described inadequate student academic preparation, mainly in the K-12 system, and to a lesser degree the changing demographic characteristics of undergraduate students, as obstacles to innovation or higher education success in general. Not surprisingly, respondents affiliated with higher education institutions (mainly administrators) were more likely to describe such obstacles (29% vs. 10%; p<.10).

“Completion [obstacle]: dealing with students who are underprepared.” President, private non-profit research university

“The declining income resources of the college going population.” President, private non-profit university

“US colleges are challenged by having to deal with a spectrum of college readiness among its applicants. That makes it hard for instructors to effectively tailor the teaching to any particular level.” Administrator, research/policy institute or think tank

“A lack of college preparedness among high school graduates impedes the ability of higher education to provide high-quality, innovative education when too much time is spent on remediation and developmental education […]” Chancellor, public college or university system

Data and Evidence

Respondents listed obstacles pertaining to data and evidence the least often, with only 17% discussing such issues. These respondents cited a variety of obstacles including the absence of adequate data, poor data infrastructures, lack of focus on rigorous learning assessments, and the challenges of measuring certain aspects of teaching and learning.

“The national data infrastructure is weak making any real reform effort largely about guess work.” Senior staff member, foundation

“Good and consistent data on completion rates, employment and income rates by school and program a few years after graduation. We need a unit record system.” President, membership association

“Lack of data-informed teaching and learning as well as limited capacity for systems to utilize data effectively.” Researcher & faculty member, public university

Appendix

Table 1. Frequency of Response to Current State of Higher Education in the US Items

Table 2. Frequency of Response to Degree Completion Rate Items

Table 3. Frequency of Response to Quality of Student Learning Items

Table 4. Frequency of Response to Affordability Items

1. All questions referred specifically to undergraduate, associate’s- and bachelor’s-degree-granting programs and to students enrolled in such programs. ↑
2. For items requesting responses on a 1 to 7 scale, respondents were presented with a slide bar and were able to select a rating at any point along the scale, including points between integers. For data visualization purposes, graphs reporting frequencies round responses to the nearest integer. ↑
3. For example, the academic and career mapping for Valencia College students intending to transfer to the University of Central Florida. See Jessie Brown and Martin Kurzweil, “Collaborating for Student Success at Valencia College,” Ithaka S+R, October 29, 2015, http://www.sr.ithaka.org/publications/collaborating-for-student-success-at-valencia-college/. ↑
4. For example, the use of over ten years of student data to generate hundreds of real-time student progress alerts at Georgia State University. See Martin Kurzweil and D. Derek Wu, “Building a Pathway to Student Success at Georgia State University, Ithaka S+R, April 23, 2015, http://www.sr.ithaka.org/publications/building-a-pathway-to-student-success-at-georgia-state-university-2/. ↑
5. For example, plans for 21 colleges from the Council of Independent Colleges to offer credit for inter-institutional online humanities courses. See Deanna Marcum and Clara Samayoa, “Leveraging Technology for the Liberal Arts: The Council of Independent Colleges Consortium for Online Humanities Instruction,” Ithaka S+R, November 4, 2015, http://www.sr.ithaka.org/publications/leveraging-technology-for-the-liberal-arts/. ↑
6. For example, student success performance funding to colleges in the North Carolina Community College System. See Jessie Brown and Richard R. Spies, “Reshaping System Culture at the North Carolina Community College System,” Ithaka S+R, September 10, 2015, http://www.sr.ithaka.org/publications/reshaping-system-culture-at-the-north-carolina-community-college-system/. ↑
7. For example, dedicated faculty development programs for University of Central Florida faculty teaching online courses. See Jessie Brown and Martin Kurzweil, “Breaking the Iron Triangle at the University of Central Florida, Ithaka S+R, August 26, 2015, http://www.sr.ithaka.org/publications/breaking-the-iron-triangle-at-the-university-of-central-florida/. ↑
8. For example, the summer bridge developmental education using adaptive learning product MyFoundationsLab at five University of Maryland campuses. See Rebecca J. Griffiths, Matthew Chingos, and Christine Mulhern, “Can Online Learning Improve Math Readiness: Randomized Trials Using Pearson’s MyFoundationsLab in Summer Bridge Programs, Ithaka S+R, December 14, 2015, http://www.sr.ithaka.org/publications/can-online-learning-improve-college-math-readiness/. ↑
9. For example, strategies for standardizing student learning outcomes assessment at the University of Pittsburgh. See Martin Kurzweil, “Making Assessment Work: Lessons from the University of Pittsburgh, Ithaka S+R, January 29, 2015, http://www.sr.ithaka.org/publications/making-assessment-work-2/. ↑
10. Percentages do not add up to 100 as some obstacles referenced more than one theme. ↑