Can You Measure It?

The debate over the value of the liberal arts rages on: in the halls of state legislatures, among faculty on college campuses, and in student newspapers. State policymakers and institutions face increasing pressure to demonstrate “value” to taxpayers and prospective students, which often means redirecting state funding to “credentials of value” as measured by students’ earnings shortly after they graduate. Yet those who have experienced a liberal arts and sciences education often describe deeply appreciating its value, even if they have difficulty quantifying that value or describing exactly which features of their education made it so meaningful.

In a meeting earlier this year, Utah state legislators on both sides of the aisle made the case for defending the liberal arts amidst proposed budget changes:

Sen. John D. Johnson, R-North Ogden, said he appreciates legislative efforts to prepare students for the workforce. “But I also am a little bit concerned that we’re blurring the distinction between a technical college and a university, and what makes a university different?” Higher education, he added, helps build a better citizenry and “thinking individuals” with the skills to lead. Sen. Kathleen Riebe, D-Cottonwood Heights, also defended liberal arts programs that could be in jeopardy. Jobs provide food and shelter, she said. “But our ideas connect us, and those ideas are important….Our universities mean more to us than just occupational optimization and efficiency rates.”

We know from our research that a liberal arts and sciences education has academic and economic value. If we also believe, as do the legislators in Utah, that it has inherent value, then we need a shared understanding of its defining features in order to promote and sustain it amid growing financial pressures. What are the curricula and experiences that produce “thinking individuals” and help us “build a better citizenry”? How do we help students develop the important ideas that will “connect us” and our communities?

At Ithaka S+R, we have spent the last few years trying to answer these questions with specificity and rigor. Building on prior scholarly work, we have come to understand the liberal arts and sciences as more than a collection of disciplines in the humanities, social and natural sciences, and arts. We also view it as a set of pedagogical, curricular, and community experiences that enrich a learner’s education. As the Utah lawmakers intuit, these experiences not only benefit individuals, they have the potential to positively impact institutions, communities, and society.

So, how do we take the contours of a liberal arts and sciences education and bring them into greater focus? Over the past four years, we have developed and tested two distinct frameworks that define a liberal arts and sciences education. First, to measure the extent to which institutions deliver liberal arts education, we published the Liberal Arts and Sciences Educational Offering (LASEO) Framework. More recently, to measure the extent to which students experience a liberal arts education, we published the Liberal Arts and Sciences Educational Experience (LASEE) Framework. For each, we developed a set of metrics which we could then apply to actual, real-world data samples to quantify the liberal arts educational experiences offered by institutions and experienced by students. In both cases, these frameworks were designed to apply to a variety of institutional settings and contexts—community colleges and four-year colleges, highly selective institutions, those that are open access, and more.

The LASEO and LASEE Frameworks, while distinct, are structured similarly, sharing the same three common core components that form the essence of a liberal arts education: pedagogy, curriculum, and community. Pedagogy captures deep engagement with content experts in and out of the classroom; curriculum captures both meaningful exposure to multiple academic disciplines (breadth) as well as learning experiences that require deep and sustained engagement in particular disciplines (depth); and community captures experiences that offer deep exposure to diverse perspectives, skills, and opportunities.

Within each of the core components of pedagogy, curriculum, and community are the attributes that comprise the component: for example, “instruction using teaching practices that foster active learning, in classroom settings that facilitate such learning” and “deep engagement with instructors outside the classroom” are both attributes of pedagogy. Together, the framework components and their attributes comprise the framework foundation.

To apply the framework to a given context using available data, we also developed a set of metrics at the institution level (for the LASEO Framework) and the individual level (for the LASEE Framework). The framework is thus designed to be flexible: while the framework foundation remains largely unchanged, the institution- and individual-level metrics can be adapted according to interest and data availability. In fact, our team did exactly that when creating two distinct versions of the LASEE Framework: the “max sample” framework, which included fewer metrics but could be applied to a much larger sample of students because it only required access to administrative data and the “max variables” framework, which included additional metrics drawn from an alumni survey but could only be applied to the subsample of students who had participated in that survey. For example, students in the alumni survey reported the number of meaningful relationships they formed with faculty members as an undergraduate, allowing for the inclusion of a rich metric that reflected deep engagement with instructors in the “max variables” version of the framework that was not possible in the “max sample” version.

Application of the LASEE and LASEO Frameworks to new settings is both possible and recommended. This could involve developing new metrics based on available data, or removing metrics that may not be a good fit in a particular context. In our prior research, we applied the LASEE Framework to more than 1.3 million students who attended public four-year colleges available in the College and Beyond II data set, but it is worth applying to other sectors (e.g., private colleges, community colleges, non-credit programs) and entire postsecondary state systems. Doing so would permit two important types of analysis: first, it would enable a bird’s-eye view of the state of liberal arts exposure across students within a given context. Second, knowing that individuals with the least exposure to a liberal arts education are precisely those who benefit most from increased exposure, developing individual-level estimates of a student’s exposure to the liberal arts enables institutions to identify those most in need and target interventions accordingly.

The state legislators, faculty, and leaders currently struggling to identify just what it is that makes the liberal arts and sciences special and worth defending would benefit from bringing data and rigor into their conversations. To that end, both the LASEE and the LASEO Framework are available for use by other postsecondary researchers and practitioners. Our team is also happy to consult or collaborate on applying these frameworks to new settings. If you’re interested in learning more about applying the LASEE Framework to your own student-level data, please reach out to Daniel Rossman (Daniel.Rossman@ithaka.org).