1. Series Overview

In Fall 2020, the American Talent Initiative (ATI), an alliance of high-graduation-rate colleges and universities committed to expanding access and opportunity for low- and middle-income students, established its newest community of practice (CoP) focused on academic equity. Together, the 37 CoP members explore topics related to creating equitable academic communities. One such area of focus is how institutions can more effectively utilize data to enhance equity-related projects. In January 2021, members participated in a webinar discussion on this topic, during which CoP representatives presented on how they have leveraged data in their academic equity work. This case study builds on a presentation given by Dr. Jill Sible, Associate Vice Provost for Undergraduate Education and Professor of Biological Sciences at Virginia Tech, titled, “Using data to fuel inclusive excellence at Virginia Tech.”

2. Introduction

Embedded in Virginia Tech’s land grant mission and its motto, Ut Prosim (That I may serve), is the commitment to providing equitable opportunities for students from all backgrounds and identities to thrive academically and socially. And while Virginia Tech has made great strides, as recognized by receiving a Higher Education Excellence in Diversity Award for five years in a row, the university recognizes there is still much work to do. [1] This is especially true in STEM fields where nationally, achievement gaps persist for many minoritized populations despite decades of federally funded interventions. Recognizing that its strong student success metrics belied some underlying disparities, Virginia Tech applied for an Inclusive Excellence grant and became part of the first cohort of grantee institutions in 2017.

The Howard Hughes Medical Institute Inclusive Excellence initiative seeks to build capacity for inclusion in undergraduate science programs at colleges and universities across the United States.[2] The focus is on populations of students who have been historically marginalized in the sciences, including racial and ethnic minorities, first-generation college students, and students who transfer from two- to four- year institutions. HHMI has challenged grantee institutions to abandon a deficit mindset, one that views these students as inherently lacking in some way, and instead turn a critical lens inward toward the institutions’ own practices and cultures. In other words, stop trying to “fix the student” and instead, “change the institution.” To achieve this goal, HHMI has equipped participating colleges and universities with rich professional development, a community of practice, reflective practices, and one million dollars each to be invested over five years.

As a large, decentralized university, Virginia Tech required a distributed approach to inclusive excellence that acknowledged the distinct cultures and needs of its many academic programs. Virginia Tech has adopted a strategy for change that empowers individual science departments across three academic colleges as the units of change. Each year, up to three departments apply to participate in the Inclusive Excellence (IE) program for two years. In year one, a team of faculty from each department engage in a year of learning, selecting from a menu of professional development opportunities that range from the theoretical (e.g., critical race theory and implicit bias) to the practical (e.g., writing an inclusive syllabus). In year two, departments engage a second cohort of faculty while simultaneously embarking on a project of their own design to impact the success and well-being of marginalized students locally within their own academic programs.[3]

In the following sections, we discuss key elements of Virginia Tech’s IE program, focusing on how we have incorporated the use of data into our efforts to fuel inclusive excellence within science departments and beyond.

3. Data as Tool, not a Weapon: Using data to facilitate learning, conversation and buy-in amongst participating faculty

A critical part of the year of learning in which participating faculty and departments engage is familiarizing themselves with data about the academic progress of their own students. A data scientist in the Office of Undergraduate Education generates fact sheets of disaggregated data for each department. The data include historical student success metrics related to retention and graduation accessed through Virginia Tech’s Office of Analytics and Institutional Effectiveness as well as real-time data accessed through the Educational Advisory Board (EAB) Student Success Collaborative Navigate platform.[4] These data are accompanied by reflection prompts to encourage departments to formulate their own questions and hypotheses regarding barriers to success for some populations of their students.[5]

We believe that the inclusive excellence challenges that we seek to address have features of both of the types of decisions that leading data scientists Foster Provost and Tom Fawcett recognize as benefiting from a data-informed approach: 1) these decisions require an element of discovery, in this case, informing faculty of the disparities that exist within their own departments, and; 2) small improvements may lead to a larger impact at scale, as long as we are intentional about disseminating what we have learned beyond a single department and our own institution.[6]

While each department has the potential to access these data on their own, we know that individual faculty often lack the time and understanding of how to access institutional data. Through the support of the IE administrative team, we seek to overcome these barriers and collaborate with faculty in the stages of the data science life cycle where they are likely to be most engaged (exploring and interpreting data) while supporting them in steps that may require the time (obtaining and scrubbing data) and expertise (modeling) of a data scientist.[7]

Unlike data that are collected to evaluate departments against scorecard or strategic planning metrics, sometimes with budgetary implications, these data are shared only among members of that department, and not circulated or compared across other participating departments. Using data as a tool, not a weapon, has become an important tenet of the IE program at Virginia Tech.

4. Collaboration and Iteration: Working together to identify specific, high-impact areas of focus for participating departments

Providing departments with these data sheets is just the first step in an interactive and collaborative process. Participants eagerly digest these data sheets to the extent that we joke that they are like candy for STEM faculty, and we often wait until we have covered all other grant-related business in meetings before we bring out the data, which inevitably consumes the rest of the meeting time with lively discussion. Faculty interaction with the data generates a host of questions and requests for additional data, thus initiating an iterative engagement between participating departments and our data scientist.

As an example, one department pressed for further information about what were the most strategic courses for intervention in their curriculum. Contrary to an expectation that foundational courses taught outside of their department (e.g., chemistry and calculus) would be the gatekeepers that determined student success in their majors, they found that some of their own courses were the most predictive of students graduating. These data helped this department apply more inclusive pedagogy to a course over which it had control. The impact of this intervention, alongside others developed in different departments, will be monitored as one measure of the success of the Inclusive Excellence program.

5. Empowerment and Reflection: Using qualitative data to bring together diverse perspectives

In order to bring student perspectives to faculty and to access faculty learning and perspectives regarding inclusivity, the IE project utilizes qualitative research in the form of interviews and focus groups. This facilitates a fully integrated approach to gathering and understanding a comprehensive picture of data on the academic successes and challenges among student populations within participating departments.

Empowering Students as Agents of Change

As part of the IE project, departments nominate students to serve as IE Fellows. These are mostly undergraduate students who are majoring in their programs and represent the identities that the department and university want to better serve. Fellows receive a stipend for participation and have access to a portion of the grant funding to initiate projects of their own choosing to promote inclusive excellence. The students’ most important role is advisory, helping the grant’s leadership and participating faculty to understand the lived experiences of minoritized and marginalized students at Virginia Tech.

An expert in qualitative research (a recent graduate from the doctoral program in School of Education at Virginia Tech) conducts individual interviews and focus groups with these students to amplify and circulate their perspectives. Rather than being treated as research subjects, the IE student fellows are regarded as co-creators of the data. We empower them to shape the direction of the resulting reports through strategies such as member checking, where they are able to review focus group reports before they are shared with IE leadership and departments.

This qualitative approach helps make the quantitative data “real” for participating faculty. To learn that a student, possibly one of their own students, experiences an environment in which they feel, “I have to code-switch. I have to go between who I am and who VT expects me to be,” confirms that implicit bias, a theoretical concept that a faculty participant will have learned about in an IE workshop, is happening on their own campus and in their own classroom. Faculty may also learn that the lower GPA of a minoritized student is not necessarily due to that student being unprepared for the rigors of college, as they may have believed, but might have something to do with the more hostile environment this student encounters relative to majority students.

Engaging Faculty as Reflective Practitioners

Faculty members also contribute to the body of qualitative data by informing the IE project at Virginia Tech. They engage in periodic, individual reflective writing and focus groups conducted by our program evaluator at the National Institute for STEM Evaluation and Research (NISER).[8] Given that faculty and departments are considered the primary units of change for this project, these data are essential in identifying short-term evidence of change in attitudes and practices as well as potential barriers to long-term impact and institutionalization of the project. At the same time, program evaluators engage faculty as reflective teachers, a strategy that can promote change in STEM instruction.[9]

Faculty perceptions on the relationship between academic excellence and diversity demonstrate that inclusive pedagogy establishes the structure and techniques to achieve academic excellence for all students. Faculty reflective writing responses highlight the need for continually improving best practices in teaching and education to best fit the needs of an ever-changing population of students.

Additionally, the impacts of COVID-19 campus closures caused IE faculty to further reflect on their own teaching styles and student access to the internet as faculty worked to make online learning equitable for all students. IE faculty focus groups revealed that students who performed poorly in their courses had the lowest engagement. In response, faculty increased email communication and incorporated online technology (i.e., Zoom, Top Hat, and Packback) to facilitate peer instruction and engagement.

An overarching takeaway from the IE faculty focus groups is that, unless the work of inclusion is an institutional expectation, particularly with regards to tenure and promotion, many of the initiatives established as part of the IE program will not be expanded or sustained beyond the lifetime of the grant. As one participating faculty member reflected:

“It [participation in IE] innately played a role in my promotion, largely because I have my two direct bosses who feel it’s extremely important and that’s trickled down. IE is definitely put forth in all of our faculty meetings as important because some of that comes from the HHMI training that we have. We talked about it at every faculty meeting so it’s being put forward to be something that’s important so as we’re promoted, it is taken more seriously.”

This feedback from faculty has prompted a shift in the emphasis of our project work to focus on faculty recognition and rewards to incentivize striving towards inclusive excellence.

Moving Forward: Early signs of progress towards inclusive excellence

We have continued to apply this same mixed methods approach to the beginning stages of evaluation of the IE program in the final years of the grant. The quantitative data may provide limited conclusive evidence of impact in the short-term, since it may take several student generations to manifest and will be compounded by additional factors (like COVID-19, and other diversity initiatives at Virginia Tech). Nonetheless, quantitative data continues to provide new participating departments evidence that they house inequities within their academic programs. All departments are situated to set their own goals and measures of success that they can monitor well beyond their participation in the grant.

Our qualitative data provide early evidence of sustainable changes toward a more inclusive environment for students in our science programs. From these data, we know that departments have embedded discussions of diversity-equity-inclusion as standing agenda items in faculty meetings. They have redesigned their curricula in ways that better connect content in sequenced courses, created trailer sections of key courses to improve time to degree, and built introduction to experiential learning and mentoring into the curriculum so that all students have access. Faculty continue to be engaged as learners, participating in inclusive excellence programming well beyond the requirements of the grant. Many are eager to engage in the scholarship of teaching and learning, bringing their own contributions to this project to fruition in the form of peer reviewed publications, an ambition we are working to support formally through the activities of the grant.

We are excited by early progress and sustained community engagement throughout the IE program. We look forward to measuring its long-term effects on inclusive excellence, participation and engagement in STEM fields among minoritized students, and student success.

Interested in learning more? For more information on Virginia Tech’s IE program, please contact Dr. Jill Sible at siblej@vt.edu. For more details on the ATI Academic Equity CoP, please contact Elizabeth Banes at Elizabeth.Banes@ithaka.org and Emily Schwartz at Emily.Schwartz@ithaka.org

The American Talent Initiative (ATI) is a Bloomberg Philanthropies-supported collaboration between the Aspen Institute College Excellence Program, Ithaka S+R, and an alliance of top colleges and universities committed to expanding access and opportunity for students from low- and moderate-income backgrounds. ATI has one central goal: attract, enroll, and graduate 50,000 additional high-achieving students from low- and moderate-income backgrounds at the nation’s colleges and universities with the highest graduation rates by 2025. For more information about the American Talent Initiative, please contact Benjamin Fresquez at benjamin.fresquez@aspeninstitute.org.

Acknowledgments

Elizabeth Banes of Ithaka S+R coordinated and contributed to this piece.

The American Talent Initiative would like to thank

  • The 37 members of the ATI academic equity community of practice, especially the staff who engage regularly with the community and lead initiatives on their respective campuses aimed at promoting greater equity in the academic experience.
  • Randall Bass, Heidi Elmendorf, and Mark Joy, of Georgetown University, and Katie Brock and Ulili Emore, of The University of Texas at Austin, for spearheading activities for the community of practice and lending their time and expertise to advancing this critical work.
  • Emily Schwartz and Elizabeth Pisacreta for their guidance and support in the coordination of this case study.
  • The staff of the Aspen Institute and Ithaka S+R who devote their time and energy to the American Talent Initiative, including Benjamin Fresquez, Mya Haynes, Martin Kurzweil, Tania LaViolet, Cindy Le, Gelsey Mehl, Yazmin Padilla, Adam Rabinowitz, and Josh Wyner.
  • Bloomberg Philanthropies, the Aronson Family Foundation, and the Gray Foundation for supporting the work of ATI.

Members of the ATI Academic Equity Community of Practice

  • Allegheny College
  • Barnard College
  • Baylor University
  • Bowdoin College
  • Claremont McKenna College
  • Colgate University
  • College of the Holy Cross
  • Duke University
  • Emory University
  • George Mason University
  • Georgetown University
  • Hamilton College
  • Hope College
  • James Madison University
  • Lebanon Valley College
  • Middlebury College
  • Muhlenberg College
  • Pennsylvania State University-Main Campus
  • Pomona College
  • Princeton University
  • Rutgers-New Brunswick
  • Stanford University
  • The Ohio State University
  • The University of Texas at Austin
  • University of California-Santa Cruz
  • University of Dayton
  • University of Illinois at Urbana-Champaign
  • University of Iowa
  • University of Pennsylvania
  • University of Pittsburgh
  • University of Richmond
  • University of Washington-Seattle Campus
  • University of Wisconsin-Madison
  • Vassar College
  • Virginia Polytechnic Institute and State University
  • Washington University in St. Louis
  • Williams College

Endnotes

  1. “Virginia Tech receives 2020 Higher Education Excellence in Diversity Award,” September 7, 2020, https://vtnews.vt.edu/articles/2020/09/VirginiaTechReceives2020HigherEducationInDiversityAward.html.
  2. “Inclusive Excellence 1 & 2,” Howard Hughes Medical Institute, https://www.hhmi.org/science-education/programs/inclusive-excellence-1-2.
  3. More information about the Virginia Tech Inclusive Excellence program as well as that of three other grantee institutions in the Mid-Atlantic region can be found at Jeremy Wodjack, Tara Phelps-Durr, Laura Gough, Trudymae Atuobi, Cynthia Deboy, Patrice Moss, Jill Sible, and Najla Mouchrek, “Learning Together: Four Institutions’ Collective Approach to Building Sustained Inclusive Excellence Programs in STEM,” in Transforming Institutions: Accelerating Systemic Change in Higher Education, 2021, http://openbooks.library.umass.edu/ascnti2020/chapter/wojdak-etal/.
  4. See the Educational Advisory Board (EAB) Student Success Collaborative Navigate platform at https://eab.com/products/navigate/.
  5. An example containing some of these data can be viewed here: https://drive.google.com/file/d/1z5j9vxsb_eD0elHVyWPJ3dSBQLJXK54V/view.
  6. Foster Provost and Tom Fawcett, “Data Science and its Relationship to Big Data and Data-Driven Decision Making,” Big Data 1, no. 1, February 12, 2013, https://doi.org/10.1089/big.2013.1508.
  7. Cher Han Lau, “5 Steps of a Data Science Project Lifecycle,” Towards Data Science, January 3, 2019, https://towardsdatascience.com/5-steps-of-a-data-science-project-lifecycle-26c50372b492.
  8. National Institute for STEM Evaluation and Research (NISER), University of Tennessee Knoxville, https://niser.utk.edu/.
  9. Charles Henderson, Andrea Beach, and Noah Finklestein, “Facilitating Change in Undergraduate STEM Instructional Practices: An Analytic Review of the Literature,” JRST 44, no. 8, October 2011, https://doi.org/10.1002/tea.20439