FEATuring YOU

Introduction

Digital tools that directly test and train individuals’ competencies are dramatically changing hiring and higher education. Technology platforms that boast robust analytic capabilities are increasingly being adopted by employers to securely and efficiently verify candidates’ fit through a wide assortment of skills-based evaluations, with 71 percent of North American employers utilizing pre-hire assessments during the screening process.^[1] In tandem, higher education has experienced a push toward online learning and alternative credentials, and as a result the field has seen a growth of non-institutional postsecondary education providers that challenge the value of the traditional academic degree.^[2]

Skills-based training and assessment technologies promise to democratize the hiring process.^[3] By automatically evaluating whether candidates possess the necessary competencies to succeed in the role to which they are applying, these tools can help eliminate human bias, diversify the talent pool, and reskill our workforce—especially appealing given that the World Economic Forum estimates a need to reskill more than one billion people in the next ten years.^[4] They are also well-suited for empowering younger learners and job candidates who are entering the workforce for the first time, unaware that they already possess many of the desirable attributes employers seek. Platforms that leverage online learning and alternative credentials can motivate these users through engaging and interactive content, allowing them to play games that test their abilities, collect badges verifying their mastery of different skillsets, and direct their own competency-based training pathways.

Southern New Hampshire University (SNHU) set out to explore this immense potential by creating a digital assessment and learning program with the specific goal of connecting marginally engaged youth, or “opportunity youth,” to employment and education opportunities. Titled FEATuring YOU,^[5] this Google.org-funded program is unique in that it is a soft skills assessment and training tool developed and managed by a nonprofit university. It uses a set of five validated, retail-based skills assessments and one game-based assessment developed in-house to test six soft skills: communication, critical thinking, customer service, adaptability, drive for results, and problem solving. Understanding that simply evaluating skills will not necessarily lead youth directly to meaningful work and education opportunities, FEATuring YOU also contains in-platform learning modules and shareable digital badges for users who master the competencies that FEATuring YOU tests and trains.

Ithaka S+R, an independent nonprofit research group, conducted a case study of FEATuring YOU to document the development and implementation of the program, understand its value-added and potential for impact in the field, and make recommendations for future developments. For its preparation, we conducted in-depth interviews in December 2019 with more than 20 individuals who have played central and supportive roles in ideating, creating, and implementing FEATuring YOU, as well as experts in skill development and digital credentials targeted for opportunity youth. In addition, we reviewed related documents provided by SNHU concerning FEATuring YOU’s development and performed a literature review of relevant publications on soft skills assessment and badging for opportunity youth.

Though we originally intended to interview both opportunity youth and mid- to entry-level hiring managers during the spring of 2020 in order to gauge their responses to FEATuring YOU, the COVID-19 crisis prevented our ability to perform such engagement. However, given the state of transformation in education and the workforce due to the pandemic, we did not want to delay sharing this case study. With the rapid shift to both digital hiring and online learning^[6]—and the loss of 26.5 million jobs and counting due to the crisis^[7]—fully-online skills-based employment and training tools are more relevant now than ever, and SNHU’s processes for designing and building such a tool can serve as an especially relevant model for the field. Towards that goal of knowledge sharing, this case study documents the various stages of FEATuring YOU: from planning to development to implementation. It concludes with key learnings from the process of creating this digital assessment and training program, providing recommendations for those in pursuit of similar initiatives.

Context

This section provides a summary background of skills-based assessments and digital badging, focusing on the promise these technology-driven solutions offer to those traditionally marginalized within the US workforce. It then gives an overview of how opportunity youth fit into these emerging hiring practices and signaling methods, arguing that soft skills in particular are fruitful avenues for upskilling young workers—particularly in the contexts of the current COVID-19 pandemic and its anticipated aftermath.

Training, Assessing, and Badging Opportunity Youths’ Soft Skills

Skills-based approaches to training and assessment for employability have been around for over a century, but the rise of big data and machine learning has ushered in a wealth of technology-facilitated assessment tools that are transforming how today’s candidates are vetted for jobs. This sea-change has the potential to democratize the hiring process, favoring job-seekers with the right match of skills and competencies for the role rather than privileging candidates with traditionally favorable backgrounds—biased prerequisites that unfairly disadvantage workers from underrepresented and marginalized backgrounds. It also allows for cost-effective approaches to learning in the form of digital training, which teaches individuals about the specific competencies a role requires, provides feedback on their performance as they evaluate their growing skillset through direct assessment, and empowers them to take ownership over their own individual learning and career pathways.^[8]

Digital badges are alternative credentials and a product of an evolving skills-based hiring ecosystem, as well as a component to a growing competency-based educational movement. Ideally, badges are portable, stackable, and sharable, and composed of a digital image tagged with rich metadata. Such embedded data describes the skills the badge signifies, as well as the criteria by which earners are evaluated—including any supporting evidence. In this way, a badge has the potential to securely and instantly verify an individual’s competencies, detailing when it was issued and by whom. Open Badges have emerged as an industry standard for how to package these information-rich visual records of achievement (Figure 1).^[9] The Open Badge Network, managed by independent nonprofit IMS Global, sets specifications for how issuers must package information contained within each Open Badge, including the criteria for how it was earned, what evidence was presented to demonstrate the achievement, and what relevant skills the badge signifies earners to possess. In this way, the protocol not only standardizes the criteria an Open Badge must make readable to potential employers or other external evaluators, it also establishes resources for its validation.^[10] One can, for example, validate an Open Badge by either entering the URL hosting it or uploading the image of the digital through the IMS Global Open Badges 2.0 Validator.^[11]

Figure 1: Metadata Embedded within an Open Badge’s Visual Emblem

Because young job candidates struggle to successfully signal their competencies to employers, and employers struggle to identify qualified candidates for jobs, digital badges have been seen by many as a potential solution to bridge this skills communication gap.^[12] Badging has especially gained interest among those who work with low-income youth and youth of color—young people who, as we discuss in more detail next, are disproportionately disconnected from work and education opportunities. Advocates for youth employment argue for soft skill development as an essential strategy for leading young people to jobs. For example, the US Chamber of Commerce Foundation reports that one of the largest barriers for youth employment is that “today’s young adults are not prepared for the workplace with adequate soft skills” and argues that employers “must tackle this challenge to help shape young adults into the strong contributors they can be.”^[13] Similarly, the US Department of Labor has addressed the need for soft skills training among American youth by developing targeted curricula for introducing workplace professional and interpersonal skills for youth ages 14 to 21.^[14]

Soft skills, sometimes called workplace readiness or employability skills, include competencies such as communication, teamwork, leadership, flexibility, adaptability, problem solving, creativity, persuasiveness, interpersonal skills, time management, and work ethic—and there is indeed a growing demand for them by employers. In fact, on the cusp of large scale automation, workforce researchers believe this trend will only continue: “Employers are and will continue to be looking for candidates with emotional intelligence, excellent communication skills, intellectual agility, a strong grasp of initiative, and a sense of ethics.”^[15] As global figures estimate that 20 to 40 percent of jobs employing youth aged 16 to 24 are at risk for automation by the mid-2030s, soft skill development will be crucial to ensuring young workers are building the competencies that employers will increasingly need.^[16]

Upskilling Opportunity Youth in the Wake of COVID-19

Opportunity youth, traditionally referred to as “disconnected youth,” are individuals between the ages of 16 and 24 who are neither meaningfully employed nor enrolled in school. The latest figures in the US (2017) estimate 4.5 million disconnected youth, with significant gaps along racial and ethnic groups. According to Measure of America—which has been studying this youth group since 2012—Native American, Black, and Latino youth face notably higher disconnection rates than their white and Asian counterparts across every income level.^[17]

These findings represent a disheartening amount of lost potential among the youngest and most underserved members of our country’s workforce. They also speak to a labor market misalignment in which employers are not leveraging the untapped capabilities of younger workers. Before the COVID-19 pandemic, national youth disconnection was in decline, dropping from a 5.8 million high in 2010 (14.7 percent) to a 4.5 million low in 2017 (11.5 percent).^[18] Unfortunately, this downward trend is unlikely to hold in the aftermath of the COVID-19 pandemic. “Young workers are typically first to be let go,” explains chief public policy officer at YouthBuild USA David Abromowitz, “particularly youth of color and those in poverty, who face the largest barriers to entering careers even in a normal economic climate.”^[19] For those same reasons, and because communities of color and economically disadvantaged communities have been the hardest hit by the pandemic,^[20] these youth are also the most likely to experience deep and long-standing disruptions to their educational and career pathways. In fact, recent data from the US Bureau of Labor Statistics show that unemployment rates for youth ages 16 to 24 have skyrocketed at a disproportionate rate during the first two months of the pandemic (Figure 2). In April 2020, youth aged 16 to 24 had an unemployment rate of 27.4 percent compared to workers 25 and older at 13.1 percent (with all workers 16 and older at 14.7 percent).

Figure 2: US Unemployment Rate, January 2000 to April 2020 (Seasonally Adjusted)

Source: US Bureau of Labor Statistics, “Unemployment in the US,” Google Public Data, 2020, https://www.google.com/publicdata/explore?ds=z1ebjpgk2654c1_.

“It might sound counterintuitive,” economist Robert Lerman recently argued, “but now—during the COVID-19 crisis—is the best time for investing in skills.”^[21] With US unemployment at its highest since the Great Depression, many Americans have found themselves under lockdown with the time and need for learning new skills. Disconnected youth are no exception to this circumstance, and there is opportunity for them and their advocates to leverage amid the rapid switch to online learning and digital credentialing. “Companies can really focus on skills-based hiring,” argues JFF CEO Maria Flynn, “They can double-down on looking at talent in new ways, and not only using a degree as a signal.”^[22] Such a change in hiring would especially benefit opportunity youth, who have historically been overlooked by employers and lack access to traditional postsecondary credentials.

As higher education is struggling to meet fall enrollment targets and prospective students are deciding that college can wait—some out of their own volition, but many others out of economic necessity—institutions must reassess how they can best serve young Americans. Scientifically validated, rigorously researched, and fully implemented tools like FEATuring YOU thus become especially interesting models of alternative learning and credentialing for the field. Distance learning could very easily remain the primary mode of educational delivery for the foreseeable future. To better serve and support their students and communities during these challenging times, institutions of higher education must innovate around their use of digital tools. And when the dust has settled, these same institutions will be tasked with preparing learners and jobseekers for a drastically different post-pandemic labor market.

Planning

In 2017, members of SNHU’s Innovation Center sought to leverage the opportunities promised by such a technology-driven solution, as discussed in the previous section, and conceived building an in-house digital assessment platform that would directly test and verify opportunity youth’s skills. This section details how SNHU began creating what would eventually become FEATuring YOU. Before delving into the specific preparations of this particular project, however, it is important to understand the environment in which it was fostered. This foundational knowledge contextualizes the project’s success—from ideation to deployment. Similarly, disclosing the parameters of the grant that funds FEATuring YOU greatly informs how it was developed and implemented. This section ends by discussing the initial research, early decisions, and formative partnerships that enabled the tool’s production, modelling such processes for organizations wishing to embark on similar innovations in higher education and workforce development.

Innovation Culture

SNHU’s push for innovation comes from the top of the organization—from SNHU President Paul LeBlanc and through the mandate of the university’s overarching mission: to “relentlessly challenge the status quo and provide the best support in higher education,” and “expanding access to education by creating high quality, affordable and innovative pathways to meet the unique needs of each and every learner.” Under LeBlanc’s leadership and toward the delivery of SNHU’s aspiration to serve diverse learners “at unprecedented scale,” SNHU founded its Innovation Center in 2015. Spearheaded by Executive Vice President of Educational and Business Transformation Wil Zemp, the Innovation Center began as the Sandbox ColLABorative, an internal consultancy and collaborative workspace for the university. Supporting the university mission and helping prepare for the future as a resource and support system for leadership, faculty, and staff as part of SNHU’s strategic commitments,^[23] the center has expanded to include SNHULabs, the research and development unit of the university, with Faby Gagne as the executive director. It is under this environment, which encourages “experimentation, failure, daring, ideation, and iteration” for “solving the problems confronting higher education,” that FEATuring YOU was conceptualized, designed, and executed.^[24] Deputy Director of Learning Science Cat Flynn serves as FEATuring YOU’s project lead and Innovation Analyst Keisha Sheedy as project manager. FEATuring You was SNHULabs’ first initiative.

The FEATuring YOU project team at the Innovation Center, guided by Vice President of Strategy and Innovation Brian Fleming, began investigating skills-based assessments early into its formation. “We were keen to understand the relationship between digital assessment and its potential for cost reduction for learners, especially those who lack access to higher education,” explains Fleming, who was the original principal investigator on the grant, “and this led us to be very curious about the assessment space as a whole and its relationship to pedagogical innovation.” This exploration was premised on the notion that digital assessment has the potential to serve as a powerful way for learners to showcase skills they already have—skills that can serve as a guide for their future educational and workplace experiences.^[25]

Such openness to innovate learning experiences is the culture under which a project like FEATuring YOU can come to fruition, and having leadership that not only encourages but also insists upon such experimentation sets a model for rapid change in a field that is often slow to welcome it. The project also aligned to SNHU’s institutional embrace of online learning. As an early player in online education, the college has grown its online enrollment from 3,000 students at the start of the twenty-first century to today’s 135,000 enrolled online learners.^[26] The concept of a skills assessment platform also mapped onto the school’s competency-based education programs, which utilize project work and direct assessments to advance learners on personalized paths through five different degree programs. By applying existing workplace skills and experiences, students can accelerate toward the degree.^[27] Designing an independent, robust tool for validating users’ marketable skillsets would thus fit within the university’s extant package of digital initiatives, in addition to its mission to serve underserved learners in novel ways.

Project Funding

In 2017, two years after its launch and early interest in skills-based assessment, SNHU’s Innovation Center proposed the creation of a unique skills-based assessment and learning tool through a grant application to Google.org. Google.org suggested that the proposal imagine the most challenging intervention in the assessment space around which one could innovate, and the team immediately identified opportunity youth as its target user base for such a project. As previously discussed, opportunity youth are one of the hardest segments to reach in the workforce development space. To be able to help this disconnected population showcase, translate, and increase their skills through direct assessment and training would promise incredible reward for this large and underserved population—a population that has been historically neglected by higher education by virtue of its non-enrollment.

SNHU’s project aimed at bridging this gap was approved by Google.org. It promised to map competency-based assessments to in-demand soft skills, facilitate job placement for opportunity youth, and deliver further learning opportunities to them free of charge.^[28] Importantly, SNHU’s proposal to Google.org emphasized the need for working directly with community-based organizations, especially those with a proven track record in engaging this often hard to reach population. It also built-in the awarding of digital badges to opportunity youth upon their successful completion of assessments, which would verify users’ competencies across a set of distinct soft skills and could be shared through social media to employers, community-based organizations, and other interested parties. These project-issued digital credentials were designed to not only be portable in this manner, but also to be eventually cumulative: “youths who complete [a] badge would…have the opportunity to ‘stack’ their experience into additional training programs, leading to another badge, micro-credential, or even degree.”^[29] As part of the project guidelines stipulated by Google.org, SNHU guaranteed that all intellectual property created from the project—including the open source code of the digital tool developed—would remain in the public domain.^[30] With these ambitions and stipulations in mind, SNHU began its first phase of research into making what would eventually become FEATuring YOU.

Market Research

By early 2018, SNHU secured full funding to develop and implement an open source skills-based assessment and training program for opportunity youth. To gather information around such an endeavor, the team held extensive interviews with relevant experts—from leaders in community-based organizations catering to opportunity youth to large regional employers and think tanks that focus on the future of work—in order to confirm that their initial idea was something truly needed in the field. After these early conversations, it became clear that none of the existing products on the market in the digital assessment space were fully customized for opportunity youth. There was a need for an assessment tool that could leverage the data capture and analytical capabilities used by advanced digital assessment vendors, utilize an engaging platform that accommodates the specific needs of opportunity youth users, and verify the competencies that matter most to entry- and mid-level hiring managers. To take an initial gauge on just what those skills might be, the team at SNHU looked to online resources offered by Emsi and BurningGlass, firms that analyze labor market data and hiring trends.^[31]

To continue their due diligence, in the summer of 2018 SNHU contracted Storyline LLC, a design thinking firm, to determine the barriers to employment specific to opportunity youth and how an assessment and training tool might help them overcome those challenges. Storyline conducted in-depth qualitative interviews with opportunity youth and coaches from community-based organizations. Several young participants expressed concern that their high school education was not preparing them for the workplace, and apprehensions about navigating the difficult hiring process were abundant. Storyline’s research also found that opportunity youth tend to struggle with conveying their soft skills to employers during the hiring process, and that their connection to employers was primarily facilitated by individuals at community-based organizations.^[32] Thus, a digital platform allowing youth to earn badges that verify their mastery of soft skills had the potential to not only connect them to employers more efficiently, but also to help community-based organizations better serve their constituents.

After incorporating the stories of opportunity youth, SNHU contracted Entangled Solutions to perform an in-depth analysis of the assessment marketplace and its intersection with opportunity youth. Using more than a dozen interviews with a diverse array of relevant thought leaders—from community-based organizations to ed-tech experts to government officials to employer recruiters—Entangled confirmed that community-based organizations are ripe intermediaries from which employers tend to recruit opportunity youth job candidates. Entangled also advised the team that the bucket term “soft skills” means different things to different employers and assessment providers, but that in general competencies such as critical thinking, problem solving, and communication were the most in-demand and that “opportunity youth score as strongly on attributes associated with the potential to advance into higher-level roles (soft skills) as other demographic groups.” Last, Entangled’s market analysis concluded that assessments using online simulations hosted on platforms with interactive user interfaces would be particularly well-suited for engaging opportunity youth users, especially when paired with the capacity to host digital signals such as badging or other alternative credentials.^[33]

Early Decisions

Following the extensive background research confirming the need and key stakeholders for their proposed soft skills training and assessment tool, the SNHU team had three overarching challenges to solve: how would they begin to design a digital assessment platform, how would they build engaging assessments targeted for opportunity youth to host on that platform, and which soft skills would these assessments test? The team formed partnerships to help answer these big questions, turning to collaborators that were mission-aligned to SNHU’s core values. This philosophical pairing would be crucial to the project’s success, and serves as a key consideration for institutions planning to engage technology providers or other external organizations when attempting to realize their planned digital innovations.

Choosing the Hosting Platform Vendor

One of the first decisions to be made concerning the creation of a skills-based assessment and learning tool, as stipulated by their funder agreement, was whether they should build or buy. After careful consideration, SNHU decided that a careful mix of both would be the best approach. Project staff decided to utilize an extant hosting platform—rather than build a costly and time-consuming digital environment from scratch—upon which to integrate tests and training materials. SNHU used six main criteria to guide their choice of assessment platform: It had to be user-friendly, audience appropriate, comprehensive, able to integrate with other tools, research-backed, and informed by learning sciences. SNHU also needed a vendor capable of issuing digital badges and meeting the requirements that stipulated all intellectual property from the project remain in the public domain.^[34] As per discovery from SNHU’s research by Storyline and Entangled Solutions, another chief criterion for selection was connecting with a platform that already had existing connections to opportunity youth and the community-based organizations that support them. Given the complexity of the project, it was also critical to work with a provider that would serve as a mission-aligned thought partner.

After careful consideration, SNHU decided to partner with LRNG, a Chicago-based organization with the goal to “ensure that all young people, especially those from underserved communities, have the inspiration, guidance, and skills needed to prepare them for life and work in the modern economy.”^[35] SNHU and LRNG shared a mutual willingness to experiment—or to “learn together,” as Wil Zemp explains. Even more, “LRNG understood the target population,” describes Faby Gagne, “That increased our interest in their product—they built things with the learners’ perspective in mind.”^[36] LRNG’s history of reaching opportunity youth across the country through an existing, extensive network of community-based organizations would be a crucial asset to the project. Importantly, the LRNG platform was not only designed to assess young people’s competencies but also to help them upskill; users can complete digital training “playlists,” test their mastery across different competencies, and then earn digital badges that verify those skills. (The details of how playlists are used in FEATuring YOU will be discussed in more detail in the following section.)

SNHU’s partnership with the platform provider proved such a great fit, in fact, that it was formally solidified in October 2018 when the university merged with LRNG. Pivotal to this successful merger was the alignment of both SNHU and LRNG’s core directives—the two organizations shared a vision to transform lives by expanding the boundaries of higher education to increase access and success for all learners—and the mutually beneficial aspects of the partnership. Director of Program Strategy and Design Michael Moretti confirms this: “The shared belief in putting the student first and creating pathways to increase upward mobility, along with the mutual commitment behind the work of both organizations prior to merging, underpins SNHU’s merger with LRNG and is core to our strategic plan as an entire organization.”^[37]

Utilizing Game-Based Assessments

One of SNHU’s earliest interests while ideating a skills-based assessment and training platform for opportunity youth was in game-based assessments. Game-based assessments introduce the element of online game playing to skills testing. This can include earning points in a scaffolded learning and assessment environment (such as in the collection of digital badges), participating in interactive videos that deliver simulated job previews through avatars, or by playing online games that assess specific skills.^[38] Findings from market research into opportunity youth’s receptiveness to skills-based assessment platforms found game-based interactive simulations to carry strong potential for engaging these target users. However, given the grant’s two year time frame and the costs involved in designing multiple game-based assessments from scratch, and since the Google.org grant required a platform that assessed multiple soft skills, the team chose to focus on measuring only one soft skill through a game-based approach: problem solving.

While there were several existing assessments on the market that measured problem solving skills, there were few game-based options designed with opportunity youth in mind. Crisis in Space, a game-based assessment for complex problem solving designed by GlassLab Games, was a very close match. Just as with LRNG, Crisis in Space’s background fit with the project’s broader mission and needs. It was funded philanthropically—from development to implementation—and with the input of hundreds of teachers during the design process. The team used Google.org funds to integrate the game into the LRNG platform.

Crisis in Space was originally created to measure collaborative problem solving among middle and high school students. The assessment was designed as a two-person, teamwork-based game where partners solve puzzles and missions to evaluate their collaborative problem solving competencies, but SNHU saw potential for individualizing and tailoring it specifically to opportunity youth. As with many of the partnerships SNHU made while making decisions about FEATuring YOU’s development, there was an immediate values-based alignment between the team at SNHU and the team that developed the game, especially with Seth Corrigan, Crisis in Space’s primary investigator and, at the time, LRNG’s director of research and development. “We met with Seth, and the matchup between values and mission were there,” explains Wil Zemp. ^[39] To modify, develop, and validate Crisis in Space as a problem solving skill assessment would be no easy feat, and having calibrated goals between the SNHU and LRNG teams would be critical to its success.

Deciding the Soft Skills to Assess

After agreeing to build the required changes and pilot the necessary validation study for Crisis in Space as FEATuring YOU’s game-based problem solving assessment, the project team began looking for additional soft skills assessments to add to the LRNG platform to make the training and testing tool more comprehensive. This would also lend credibility to the game-based assessment, give opportunity youth the ability to demonstrate more competencies to employers, and allow SNHU to meet the grant outcomes set by Google.org.

Market research conducted by Entangled Solutions identified the assessment provider Core Score as being particularly well-suited to the goals of FEATuring YOU. Core Score is a soft skills assessment for entry- and mid-skill workers that “measures core competencies and can be used to articulate the potential gaps in a worker’s experiences and knowledge, assuring proper alignment to training opportunities”;^[40] it is designed by Innovate+Educate, a firm with the mission of “creating new employment pathways for workers based on skills and competencies.”^[41] One of its scientifically validated products is its Entry-Level Core Skills Assessment (CSA-E), which assesses five of the six entry-level skills identified by the National Retail Services Initiative’s Competency model: communication, drive for results, customer service, adaptability, and critical thinking.^[42] Beyond being rigorously validated, CSA-E was attractive to the team designing FEATuring YOU because retail jobs are those that opportunity youth most commonly seek. Having made this final overarching decision, the project team was ready to begin integrating FEATuring YOU’s different components onto the LRNG platform.

Development

FEATuring YOU’s chosen assessments and learning materials needed to be significantly tailored to meet the needs of opportunity youth users. This would require extensive component customization, iterative product development, and repeated user testing. Such development processes would involve extensive cross-organizational collaboration—from various members of the SNHU, LRNG, and Core Score teams. And while these different stakeholders were especially well-aligned in terms of their collective desire to build an engaging online training tool for marginalized learners and their shared commitment to building such an innovation in fast-paced production cycle, detailing FEATuring YOU’s development process illustrates the labor-intensive nature of digital innovation. Similarly, SNHU’s creation of digital badges to verify users’ mastery of six soft skills can serve as an instructive case for educators considering experimentation with online credentials. This section thus describes how such developments were made, spanning mid-2018 to the project’s launch in 2019.

Building the Key Components of FEATuring YOU

There are four key components of FEATuring YOU: the game-based problem solving assessment Crisis in Space, the five Core Score retail soft skill assessments, the Level Up Playlist training modules, and the SNHU-powered digital badges. Each constituent part is hosted on LRNG’s digital platform, accessible through the FEATuring YOU landing page (Figure 3).

Figure 3: FEATuring YOU’s LRNG Landing Page

Crisis in Space: A Game-Based Assessment for Complex Problem Solving

By far, Crisis in Space was the most complex element to develop for FEATuring YOU. While chosen during the planning stage for its ability to meet the unique needs and preferences of opportunity youth users, the original game still required significant changes. First and foremost, it needed to be changed from a two-player to a single player game. During this process, several new puzzles were added to replace its prior emphasis on collaborative problem solving with that performed independently.^[43] The design of the new game-based assessment was also informed by a much broader goal: allowing users to earn college credit directly through FEATuring YOU. To this end, the Crisis and Space development team modified extant assessment puzzles and created new ones that evaluate players’ problem solving skills as specified by the corresponding Valid Assessment of Learning in Undergraduate Education (VALUE) rubric created by the Association of American Colleges and Universities (AAC&U).^[44] VALUE rubrics provide a framework for assessing student work “to determine whether and how well students are meeting graduation level achievement in learning outcomes that both employers and faculty consider essential,” and have been approved as meeting the national accountability standards used in regional accreditation.^[45] This means that the successful completion of the Crisis in Space assessment verifies a user’s ability to design, evaluate, and implement strategies for answering open-ended questions or achieving a desired goal.^[46]

Throughout its iterative development, Crisis in Space underwent a two-phased user testing and validation process, which would lead to significant game redesign. Phase 1 involved recruiting several dozen individuals aged 18 to 24 who neither held a college degree nor were currently enrolled in school; these participants played the game while following a think-aloud protocol that enabled SNHU staff to collect real time feedback.^[47] Recorded observations of these sessions enabled developers to test whether or not the game elicited the desired problem solving skills and tactics it was designed to measure, a process called construct validation.^[48] Phase 2 applied learnings from Phase 1 and also occurred before Crisis in Space was integrated on the LRNG platform; around 100 young users were recruited to play the game and provide feedback through a structured Qualtrics survey on their experience.

During testing, users reported feeling absorbed in the game, noting that they would recommend it to others, and most also suggested that the game was indeed testing problem solving skills that were likely valuable to employers.^[49] However, there was also feedback regarding the game’s difficulty and lack of transparency. In its first iteration, Crisis in Space asked users to work through several puzzles, each with varying degrees of difficulty, testing their problem solving abilities through performance-based online simulation. For example, users were asked to complete different space-related missions such as designing an ideal workout routine for astronauts given a certain set of restrictions or turning on emergency course correction equipment during an asteroid shower given a complex set of rules (Figure 4). Due to the desire to make the assessment rigorous enough to pass the initial scrutiny of both employer partners hiring candidates and academic offices conferring credit, these early puzzles were quite challenging, with the difficulty level increasing quite dramatically after the first mission. ^[50]

Figure 4: Sample Problem Solving Puzzle from Crisis in Space

SNHU thus adjusted Crisis in Space to make it more user-friendly and engaging for opportunity youth by editing instructional language, designing additional puzzles with lower difficulty levels, and providing feedback to players about their performance after gameplay. Now, in the newest version of Crisis in Space published on LRNG in December 2019, upon completion, users are shown their scores across skill trees that map out specific capabilities—such as how well they can identify strategies or implement solutions when solving problems. They are also provided feedback on how to improve within these skill areas—such as implementing multiple solutions and paying more attention to contextual factors when solving problems.

Core Score: A Suite of Retail Soft Skill Assessments

While the game-based problem solving assessment was under development, SNHU was also adding the previously mentioned Core Score retail assessments to FEATuring YOU. Compared to the necessary development and validation that Crisis in Space required, tailoring Core Score and integrating it into the LRNG platform would be much simpler. For instance, unlike Crisis in Space, Core Score’s assessments have already been rigorously validated to test the skills of entry-level retail employees. To add Core Score to FEATuring YOU, SNHU first licensed its full Entry-Level Core Skills Assessment (CSA-E) product and then developed an API that broke it into different components. While users who take CSA-E outside of the FEATuring YOU program are given a singular score across a diverse set of competencies, FEATuring YOU segregates its five chosen soft skills into shorter standalone assessments users can complete independently (Figure 5): communication, critical thinking, customer service, adaptability, and drive for results.^[51]

Figure 5: The Core Score Critical Thinking Assessment and Playlist

On the LRNG platform, these five soft skill areas are referred to as “playlists.” Each playlist begins with an initial assessment designed to distinguish high-performing potential employees from lower-performing ones.^[52] These are forced-choice assessments—meaning that users select responses from a list of supplied answbuilders (i.e. multiple choice, true or false, or ranking formats) —that utilize situational judgment questions—meaning that users are asked how they would handle typical on-the-job situations.^[53] When a participant answers these questions satisfactorily, they master the skill being assessed. In this way, a FEATuring YOU user might master one competency while needing further development in another. If a user does not pass an assessment on the first attempt, they are then directed to a Level Up Playlist to help them master that particular competency.

Level Up Playlists: Soft Skills Training Modules for Opportunity Youth

Through early conversations with community-based organizations and initial testing of opportunity youth users, the SNHU team realized that while some users might be able to immediately pass an assessment, there needed to be content available for those who did not master that competency at first pass.^[54] The need for upskilling content was met by using Level Up Playlists. Similar to building a playlist on a music platform, curating playlists on LRNG allows users to build their own learning experience.^[55] Users across the LRNG platform can complete playlists to master disparate in-demand hard and soft skills—such as coding, career readiness, and music production—and each playlist includes an element of skill assessment—such as completing short quizzes, submitting an essay, or composing a piece of music.^[56]

For FEATuring YOU specifically, each of the five integrated Core Score assessments were given a distinct Level Up playlist. The playlists are only recommended to users who do not pass the initial quiz on the first try, meaning that not all users will interact with this part of the platform. The Level Up playlists include videos and mini-assessments meant to help users gain mastery in that particular soft skill. Each playlist has anywhere from three to five videos that are required to view before a user can complete the playlist. Once users complete the Level Up Playlist, they are then invited to retake the initial Core Score assessment of that module. Users who then pass the assessment after retaking it are awarded an SNHU-powered badge, signifying their mastery in that specific soft skill.

Powered by SNHU: Digital Badges Verifying Soft Skill Mastery

Like most institutions that serve adult learners and are involved in the online education space, SNHU had previously delved into the world of alternative credentials before starting work on FEATuring YOU. This initial exploration led the team to examine the potential of digital badges, specifically because of their opportunity for scale.^[57] This promise, combined with the fact that the LRNG platform was already configured to support badges, made them a natural choice for the new soft skills assessment and training tool. Awarding digital badges was also attractive since they could be easily shared on various social media sites—a way to keep opportunity youth users engaged with FEATuring YOU’s content after leaving the LRNG platform. The team choose to follow the standards set by the Open Badges initiative (outlined in the “Context” section), as this infrastructure meets the open source requirements set by Google.org’s grant.

FEATuring YOU awards users SNHU-powered Open Badges. Badge earners who master one of the six soft skills areas are awarded a visual image signifying their achievement, branded prominently with SNHU’s logo. Over the last decade, the distinctive blue and yellow label has been featured in SNHU’s marketing strategy, campaigns which have greatly expanded the University’s brand recognition. During this time, SNHU has managed to transform its regional reach into a national footprint through cable TV ads, social media campaigns, and workforce partnerships.^[58] This matters for FEATuring YOU in that SNHU’s national brand lends authority to the digital badge it endorses, providing a currency many digital credentials lack. FEATuring YOU’s badges also clearly display the metadata embedded within them: The name of the earner, the date they earned the badge, the criteria the badge signifies the earner to have demonstrated, and the evidence the earner submitted to verify their mastery of the particular skillset assessed.

Each badge awarded through FEATuring YOU’s six soft skill training modules is hosted on a unique URL published on the LRNG platform (Figure 6).^[59] That web page displays the Open Badge image (i.e. Problem Solving), the criteria its named earner has demonstrated in earning it (i.e. an ability to consider many details when implementing solutions), and even the evidence that earner submitted to earn it (i.e. the earner’s specific problem solving skill tree as evaluated by Crisis in Space). Users can thus prove their mastery of an in-demand competency by simply linking that permanent address on their resume or sending it directly to potential employers and other external evaluators. Even more convenient, badge earners can share their information-rich digital credential through more than 150 social media and digital service integrations: from Gmail to Facebook to LinkedIn to Twitter. More visually-minded earners can also download their badge, and although not discernable on that generated image, the file still contains all Open Badge compliant metadata This can even be quickly and securely verified through the Open Badges Validator.^[60] On the LRNG platform, these badges can be combined to form stacked meta-badges, allowing users to create an entire portfolio of work and digital credentials.

Figure 6: FEATuring YOU Problem Solving Badge and Metadata

Connecting with Potential Employer Partners

While the development of the platform was crucial to the project’s success, SNHU also needed to ensure buy-in from employer partners. In November of 2018, SNHU held a webinar with three potential employer partners describing the goals of the project. The attendees were selected through pre-existing relationships with SNHU and were invited to share their thoughts and feedback regarding FEATuring YOU—particularly whether or not young users who completed the program’s online assessments would in fact possess the basic skills required to succeed in a frontline position.^[61] These employers were also asked to commit to interviewing opportunity youth users who demonstrated mastery of certain soft skills by completing FEATuring YOU’s assessment and training modules and earning a SNHU-powered badge.^[62] While many employers voiced that FEATuring YOU was an attractive concept in theory, they noted multiple barriers for adopting its assessments and badges into their current hiring procedures. Because it is so difficult to change companywide screening and vetting practices, they were unsure how to incorporate a new and experimental tool.^[63] However, this conversation was essential: Engaging with employers directly helped the SNHU team conceptualize individual companies’ specific needs and nuanced requirements and informed how they might partner with employers for the launch of FEATuring YOU.

Implementation

Having built FEATuring YOU and integrated its different training, assessment, and badging materials onto the LRNG platform, the SNHU team leading the project was ready to roll-out its unique soft skills tool. This section summarizes that launch, which began in early 2019, and its further implementation to date. It begins by describing the events SNHU hosted to introduce opportunity youth users to the online program and digital badges, and then discusses events held later in 2019 with a retail partner. Because outreach was critical to driving FEATuring YOU’s implementation, we then look at the various marketing strategies SNHU created for recruiting opportunity youth participation. An overview of the ongoing longitudinal study designed and managed by the project team—which is still ongoing, though on pause while SNHU navigates the COVID-19 pandemic—closes this section.

FEATuring YOU’s launch exposed the tool to a diverse set of community-based organizations, opportunity youth job candidates, and retail employers’ hiring managers—audiences that are problematically excluded from higher education’s dialogs on online learning. By inviting new voices to the conversation and listening to their impressions of the digital training and assessment tool, the team was able to make informed decisions about altering certain strategies that proved less effective than originally anticipated. Rather than remain beholden to preconceived ideas about how different audiences might react to FEATuring YOU, the team at the Innovation Center tried to pivot when project plans needed to change to better reach its desired opportunity youth users and the adult intermediaries connecting them to employment. Institutions looking to innovate in the digital learning space—especially when developing tools for an audience that has been historically relegated to the fringes of higher education—should embrace the strategic flexibility SNHU practiced during FEATuring YOU’s implementation. Though it does not necessarily guarantee optimal outcomes or deliverables, making room to revisit project goals and adapting methods for pursuing them can be a helpful means for recalibration. This includes continuous reflection on lessons gained during early deployment—both successes and failures carry powerful insights that can push innovation forward.

Launch Events

After FEATuring YOU was launched on the LRNG platform in February 2019, the project team began organizing a number of in-person events to help youth engage with its soft skills assessment and training offerings. These in-person events were held in Boston and Chicago, where SNHU and LRNG had existing connections with community-based organizations. In some cases, these events also offered youth the opportunity to interact directly with interested employers. For example, at a Boston pop-up event planned directly with and for the Boston Boys and Girls club, more than 23 assessments were completed with at least three young adults subsequently referred to a participating retailer’s human resources department.^[64]

Importantly, these initial in-person events planned for the specific needs and preferences of their young attendees. For example, knowing that disconnected youth might not have access to a personal computer, SNHU provided the requisite technology needed to ensure participants could engage with FEATuring YOU—by reserving onsite computers in student centers or by loaning users university laptops during the event. The project team also worked to create an encouraging, low-stress environment for young people who might be unfamiliar with the hiring process and employment in general; this included providing participants food and awarding them prizes. As these young users interacted with the tool, SNHU staff not only observed how they engaged with its different assessments, Level Up Playlists, and digital badges, but were also attentive to address any concerns and frustrations. Project team members fielded young individuals’ questions, and they fostered a motivational environment that encouraged participants to continue interacting with the training modules even if they did not master the skill area at first try.^[65]

During these introductory events, young attendees were compensated with $25 Amazon gift cards for lending their time to SNHU and giving their feedback on FEATuring YOU. This signaled to young job candidates that their intellectual labor was valued. It also demonstrated that although they may be entering the workforce for the first time—and thus struggling to identify their qualifying capabilities—they already possessed relevant knowledge and experience for which organizations would gladly pay to access.

Retail Employer Events

After hosting the series of pop-up launch events, the FEATuring YOU team sought to create direct pathways to work—connecting opportunity youth to local employers currently hiring. In May 2019, SNHU found a partner in retailer Target, and the two organizations held a job event in Boston during which young candidates completed three FEATuring YOU soft skills assessments Target identified as the most in-demand for its roles: communication, customer service, and adaptability. Users who mastered these competencies and earned their corresponding Open Badges were then able to speak with a Target human resources representative and discuss potential employment opportunities. Over the course of the night 30 badges were awarded; both candidates and hirers voiced their enthusiasm for the tool, noting that the emphasis on soft skills was particularly meaningful in connecting young participants to jobs at the retailer’s area locations.^[66] In fact, SNHU and Target continued this collaboration, hosting additional job skills events in the Boston area and connecting more local youth to retail employment opportunities.

Opportunity Youth Outreach

In an effort to reach a large number of opportunity youth across the country, SNHU internal marketers worked with the FEATuring YOU project team to design a variety of online advertising initiatives. These digital campaigns encouraged young users to register for LRNG and interact with FEATuring YOU independently, and were mainly published through social media channels. However, these interactive marketing initiatives proved unfruitful. Outreach efforts then focused on print media: Materials such as brochures and handouts were developed to recruit youth for local pop-up events and increase these recruits’ engagement with the platform during the events. After engaging with opportunity youth in-person, the team again regrouped to shift their messaging. They learned that this user group was motivated by material outcomes and specific rewards rather than abstract opportunities or future projections. There was a disconnect between FEATuring YOU’s various online materials and any explicit benefit they might hold for users’ post-event. Much of this came from confusion about digital badges: How do they work? How are supposed to use them? What do employers do with them? To help opportunity youth conceptualize badges’ real-world application and long-term utility, SNHU created an explainer video on digital credentials and soft skills—its third communication strategy for engaging disconnected youth.

Social Media Campaign

Social media ads on Instagram, Twitter, and Facebook were piloted in October of 2019 and were run nationwide. The ads often included a link where youth could sign up to use FEATuring YOU through the LRNG platform. The campaigns used geotargeting and self-reported data on Facebook and Instagram to identify youth that were 16-24 years old and were not currently enrolled in school nor had previously completed college.^[67] The team expected this to yield a large number of registrations as 81 percent of US adults are online daily, with large shares of younger adults aged 18 to 29 reporting to be constantly connected to the internet.^[68] While data showed that youths were clicking on the initial ad, many left before even completing the sign-up process. The team partially addressed this issue by providing compensation for completing registration.^[69] While compensation did decrease sign-up abandonment, the team still found that the social media campaigns did not yield as many account registrations as they had anticipated. For this reason, the FEATuring YOU team decided to shift their marketing strategy away from social media and towards working directly with community-based groups to host in-person events with opportunity youth.^[70]

Print Marketing

The Innovation Center’s creative team initially designed print marketing tools to help bring opportunity youth to FEATuring YOU events. The team designed brochures, postcards, and handouts explaining FEATuring YOU’s longitudinal study (described in detail below), digital badging, and the value of attending SNHU’s soft skill assessment and training events. To increase engagement with opportunity youth, a traditionally under-targeted and hard-to-reach demographic, the team made sure to use language and visuals that would not only appeal to but also encourage them. This included messages urging young individuals to apply for jobs, even if they thought they might not have any relevant skills to warrant being hired: “you have the skills [employers] need, and they want you to apply for these jobs.”^[71]

As with the social media campaigns, the team used learnings form these initial materials to adjust their approach. For example, after a Boston pop-up event, the team took note of specific language that was not appealing to young attendees.^[72] They also learned that leveraging relationships with community-based organizations for these materials’ dissemination is a good strategy for amplifying reach. Providing brochures and handouts to local organizations and even in public spaces like libraries is one way the team plans to spread the word about FEATuring YOU.^[73] The team plans to engage in similar grassroots approaches to marketing the project.

Badging Video

Early user testing of FEATuring YOU indicated that many young people did not understand what a digital badge was or how it could help them secure employment. To address this, the team developed a badging video to bridge the gap between the project’s audience and the project itself. The video was designed to appeal to youth ages 16 to 24 and had to be densely informative while remaining short and entertaining.^[74] To achieve this balance the team decided on a 90-second videogame-styled recording that would explain the purpose of badges and their usage on social media and resumes. The video included language that aligned with that from LRNG and SNHU’s Digital Credential Lab on what a badge was and how it could be used. Coming up with this definition of a badge as a credential was critical and was a large part of the work that went into the badge explainer video as a whole.^[75]

Explaining a concept like digital badging to opportunity youth is no easy task, and while this video was used at events, it remains to be seen how it will help opportunity youth engage with FEATuring YOU. An informal critique from community-based organization staff who work with opportunity youth, for example, has centered on the video’s use of career development jargon that is unfamiliar to many young people who have never before pursued employment. This carries potential unintended consequences such as deterring engagement with something unfamiliar and causing feelings of misfit or ineptitude with opportunities for which they are well-suited and qualified. The video has not been tested explicitly with opportunity youth to see what language might be more or less helpful to them, which would likely yield a video that is perhaps less technical and more relatable for the population to which it is trying to appeal.

Longitudinal Study

Where feasible, youth participants who attended FEATuring YOU events were simultaneously participating in an IRB-approved longitudinal study of the soft skills training and assessment platform. This study allows the SNHULabs team to analyze opted-in users’ engagement data and skills development. With the ultimate goal of tracking participants’ employment outcomes, participants are then invited to complete a four-month follow-up survey distributed through Mosio,^[76] a company that uses two-way text messaging to collect data from study participants.^[77] Grounded in grant-related outcome measures, the survey asks participants whether or not they were able to secure employment, and if so, whether they were able to retain employment for 90 days. For youth who were already employed, the survey asks if they were able to obtain a promotion in the time since they had completed FEATuring YOU. The longitudinal study is presently ongoing, and the team has continued to recruit participants and administer follow-up surveys (through an automated process) through July 2020 in order to fully capture those engaging with the survey last fall and winter. As data collection is concluding, analysis is underway.

While the grant only specifies that the team measure outcomes from the tool as a whole, the researchers on the team are interested in more granular data that can show which assessments specifically led to improved outcomes.^[78] The team is interested in correlational examinations between the various assessments as well as information on how many attempts were needed to complete the various assessments. These correlations as well as any other more granular data can be used to improve FEATuring YOU even more going forward. Since the project had to comply with IRB guidelines surrounding minors, only youth 18 or older are invited to participate in the study unless there is the opportunity to secure parental consent for minors. This means that platform engagement and follow-up survey data will be primarily collected for youth 18 and older even though the completion rate is actually greater for those who are under 18.^[79] It is possible that the study component itself contributes to lower completion among adult participants, as staff report that the long informed consent form is tedious for participants. While the full array of data on this population is not yet available, some preliminary results are presented below.

Outcomes and Next Steps

This section addresses FEATuring YOU’s preliminary outcomes, results from its ongoing longitudinal study, as well as the future plans SNHU has for its next phases of development and implementation.

Preliminary Results

FEATuring YOU has exceeded its primary project milestone, achieving over 2,400 soft skill assessment completions overall. Hundreds of youth throughout the US have engaged in the program, and the total users that have visited FEATuring YOU is 1,054. Of these, 64 percent completed an assessment and received a badge—the most frequently earned being communication—and 14.5 percent engaged with a Level Up Playlist.

The program entails two research studies: a longitudinal employment outcomes study and a validation study of the game-based assessment, Crisis in Space. To date, the longitudinal study has 236 participants opted-in, and the text message-administered follow-up survey has a current response rate of 12 percent, with 19 percent of participants starting the survey.^[80] Crisis in Space is still undergoing its validation study, but to date this study has included 249 participants. The team has also recently deployed a Level Up Playlist for this the problem solving module, which has been integrated into the LRNG platform.

Future Plans

There is a wealth of possibilities to pursue for FEATuring YOU’s next stages of development and implementation. The team has been granted a no-cost extension through the end of January 2021, and is looking for ways to further the project over the next few years. Obtaining a no-cost extension allows the SNHU team to continue progress on analyzing employment outcomes data, validating the game-based assessment of problem solving skills, disseminating research study results, and developing a future-oriented roadmap.

Moving forward, the SNHULabs team intends to build on the learnings from FEATuring YOU and develop a game-based assessment of oral communication skills. One critical component missing from FEATuring YOU was preparation for interviews, which according to employers is where many opportunity youth struggle to demonstrate their skills.^[81] This new product will provide dynamic feedback from audio recordings and transcribed text to help learners finesse everything from their timing and expressiveness to the syntax they are using during interview scenarios. While the goal is to create an ecosystem solution for assessing oral communication, the team is starting by developing a minimum viable product of a web browser game. The possibilities are endless, provided SNHU can secure additional funds.

Lessons Learned

FEATuring YOU has immense potential for reaching opportunity youth, helping them showcase their soft skills, training them toward employment, and connecting them to work and education opportunities. It also stands as a model for what can be achieved in a higher education setting that rewards taking big risks in order to fulfill its mission, and the types of challenges that can be anticipated along the way. This section synthesizes key themes uncovered by our research into FEATuring YOU, drawing from examples specific to the goals and components of SNHU’s soft skills assessment, training, and badging tool for opportunity youth. However, our analysis of these key findings can be extended as thematic guides for other organizations embarking on the creation of an ambitious digital project.

Higher education institutions, employers, and individuals alike are struggling to adapt amid the shock of a global pandemic. But as centers of learning and service, American colleges play a critical role in shaping our society. Despite its deep challenges, the current circumstances present an unprecedented opportunity for the field of higher education at large—and colleges and universities in particular. Institutions that leverage the collective sea change, embrace emerging digital environments, and adopt a cultural of innovation can help our society heal, beginning with forging pathways toward a brighter future for our most disadvantaged members.

Select Mission-Aligned Partners and Technology Vendors

When SNHU began making early decisions around building FEATuring YOU, the team behind the project recognized the need to collaborate with experienced individuals with deep knowledge in both skills-based assessment technology and opportunity youths’ specific challenges in gaining meaningful employment. In choosing who to partner with, the SNHU team was very careful to engage partners who not only shared but practiced the university’s chief directive: “By relentlessly challenging the status quo and providing the best support in higher education, Southern New Hampshire University expands access to education by creating high quality, affordable and innovative pathways to meet the unique needs of each and every learner.”

Our research into higher education-focused networks emphasizes the power of mission-alignment in building successful collaborations, especially for intensive, high-capacity projects with ambitious goals.^[82] FEATuring YOU is one such project, and the SNHU team behind it repeatedly engaged partners that shared the same primary directive and grounding philosophy around innovation and learning. This began when choosing which vendor would host the digital learning and badging platform, and eventually led to a merger between LRNG and SNHU due to such excellent fit. And it continued as the team selected assessment providers, engaging two teams of scientifically rigorous evaluators who are especially focused on how skills-based learning and assessment tools can empower marginalized young people. Relatedly, the team’s ability to draw on the talent and expertise of other SNHU staff across the organization as needed served as a significant resource for the project; it allowed the team to efficiently troubleshoot or execute new tasks with the support of dedicated individuals equally passionate about the project and SNHU’s overall mission. Organizations looking to innovate within digital learning should also recognize, then, the need for intra-institutional alignment. Buy-in from faculty, learning designers, academic deans, and registrars will be especially important for credit-conferring tools.

Engage your Target User Early and Often

FEATuring YOU faced a notable challenge in engaging with opportunity youth, the particularly hard-to-reach population the platform targets. While background research was done with opportunity youth users before the project was integrated onto the LRNG platform, there was a lost opportunity to continuously re-engage with these target users during the design process. For younger users, such engagement is especially beneficial: Research has shown that youth-involved co-design of badging systems empowers young learners and increases the impact of their learning experiences.^[83] Some unintended consequences for FEATuring YOU included: a confusing use of jargon in both marketing and learning materials; insufficient guided support for this user group on how to approach the FEATuring YOU material on the LRNG platform (and in which order); and missing elements that help connect youth to work after their skills are evaluated, such as tips on resume building, job searching, and navigating the job market.

Developers of similar programs should prioritize the involvement of their target users during each stage of the iterative design process. Ideally this is done not only during user experience design and testing—which the SNHU team did successfully while building Crisis in Space—but in all elements related to production and launch. Team members from marketing, learning design, and product development can best connect with their target users if they are able to interact with them directly, tailoring their work to these learners’ unique needs. Those considering a digital project for a user group similar to that which SNHU targeted should remember that disconnected populations will require creative strategies for engagement. Opportunity youth are a problematically understudied and underserved group, and the FEATuring YOU team returned to the drawing board several times when previous outreach initiatives delivered suboptimal results. Such persistence will be necessary for others planning to support and target this group. The general lack of data and literature on these young people poses an immense challenge for educators and advocates aiming to provide them with meaningful learning and employment experiences, which makes continuous engagement with them all the more essential.

Push for Cross-Sector Buy-In and Digital Credential Transparency

Alternative credentials such as badging hold the potential to democratize the hiring process and empower learners to design their own competency-based pathways economically. “However,” researchers remind us, “the implementation of badges is growing considerably faster than research on their effectiveness at motivating, rewarding, and connecting learning.”^[84] Because these digital signals are new and are competing in an unregulated market, their cultural currency has yet to be determined. For example, both young FEATuring YOU users and their adult advisors have struggled to discern their full potential, unsure what value earned badges will have external to the LRNG platform. Employers also share this complaint surrounding digital badges at large, as it can be unclear for many what the earning of a badge actually confers. When confronted with a digital credential, audiences need to know the answers to many immediate questions: What skill does this emblem verify the user as having? What assessment was provided to test that skill? What kind of quality assurance is overseeing this evaluation process?

FEATuring YOU issues credential that are Open Badge compliant, and thus conform to a standard of informational transparency, such as criteria demonstrated, evidence submitted, and detailed skill descriptions. While this information helps earners contextualize their own Open Badges, there is still a translational gap when sharing such information with external audiences. This, of course, runs counter to the very purpose of earning a badge in the first place—to signal the possession of a specific competency. Hiring managers often do not have the time or capacity to click through external web pages hosting disparate digital credentials during the resume review. In fact, the process is more likely than not automated through enterprise management software and applicant tracking systems. The large burden for ascertaining the meaning contained within a badge thus remains an incredible barrier to widespread adoption. Though nonprofits like Credential Engine are working to increase digital credentials’ transparency and literacy,^[85] it is still unclear how employers will integrate them into the screening process.

Institutions that are interested in experimenting with digital credentials should think through how earners and evaluators—whether potential employers or credit-issuing entities—can easily assess the meaning behind a badge. Is the digital credential primarily a motivational tool that helps users work toward a set of competencies? Is the goal to use it for gaining employment? Can stacking credentials lead to earned credit? Each directive need not be mutually exclusive, but recognizing the flooded marketplace for digital credentials—measuring more than 730,000 in the US alone according to a 2019 report^[86]—is an important first step when entering into this space. However, all stakeholders will have to collaboratively push for transparency to establish cross-sectional buy-in: badge earners, badge issuers, and the millions of employers who continuously voice a need for workers with the very skills these badges verify earners to already possess.

Embrace the Move to Online Learning with a Focus on Digital Equity

The physical realities engendered by the current COVID-19 pandemic will dramatically change the future of how we learn and work. Not only have schools rapidly adopted online strategies, but so have employers: “Every major company is accelerating their digital transformation at lightning speed.”^[87] There are certainly methods for embracing this seedbed for tech-driven innovation that do not exploit the current crisis; developing tools like FEATuring YOU that help uplift traditionally marginalized learners during this economic reset is certainly one such strategy. Gathering community feedback during the pandemic, the National Youth Employment Coalition (NYEC) has noted that many organizations are struggling to put their job-readiness curriculum online through virtual platforms. As higher education was quick to move online, partnering with youth-oriented organizations in the community could make an especially well-suited collaboration for colleges looking to extend their reach. There are regional labor market benefits to this as well: Upskilling displaced workers and unenrolled students matters now more than ever, and digital assessments have the ability to efficiently pair qualified candidates to jobs in new and necessary ways. In fact, despite the crisis, local retailers are currently hiring.^[88] Tools like FEATuring YOU can help connect unemployed individuals to these retail jobs.

As online learning and digital hiring expand, it is imperative that extant and future platforms address issues of digital equity. This begins with ensuring reliable internet access for low-income and marginalized communities. NYEC, for example, has noted that a key struggle during the pandemic’s move online is that many young people lack broadband internet.^[89] Organizations such as EveryoneOn have prioritized this well-before the pandemic; their “Low-Cost Internet Service Programs Tool Kit” for K-12 should be tailored and adopted by higher education institutions in order to ensure that their learners are able to receive needed internet resources.^[90] As a field, higher education must work to ensure that the move online spurred by the pandemic does not widen an already deep divide. Digital tools must actively work to increase rather than limit educational equity.

Endnotes

1. “2019 North America Candidate Experience Research Report,” Talent Board, March 30, 2020, https://3cmsd11vskgf1d8ir311irgt-wpengine.netdna-ssl.com/wp-content/uploads/2015/05/2019-NA-CandEs-Research-Results-Report_03-30-2020.Final_.pdf. ↑
2. Sean Gallagher, ”Educational Credentials Come of Age: A Survey on the Use and Value of Educational Credentials in Hiring,” Center for the Future of Higher Education & Talent Strategy, December 2018, https://www.northeastern.edu/cfhets/wp-content/uploads/2018/12/Educational_Credentials_Come_of_Age_2018.pdf. ↑
3. Meagan Wilson, “Uncharted Territory: How Employers Are Using Big Data and AI to Redesign the Hiring Process, Ithaka S+R, December 11, 2018, https://sr.ithaka.org/blog/uncharted-territory-how-employers-are-using-big-data-and-ai-to-redesign-the-hiring-process/. ↑
4. Saadia Zahidi, “We Need a Global Reskilling Revolution – Here’s Why,” World Economic Forum, January 22, 2020, https://www.weforum.org/agenda/2020/01/reskilling-revolution-jobs-future-skills/. ↑
5. “Future Employment Assessment Tool (FEATuring YOU),” LRNG, https://www.lrng.org/o/future-employment-assessment-tool-featuring-you. ↑
6. Louis Columbus, “Remote Recruiting in a Post-Covid-19 World,” Forbes, March 30, 2020, https://www.forbes.com/sites/louiscolumbus/2020/03/30/remote-recruiting-in-a-post-covid-19-world/#60901b0d599f. ↑
7. Lance Lambert, “Real Unemployment Rate Soars Past 20% – And the US Has Now Lost 26.5 Million Jobs,” Fortune, April 23, 2020, https://fortune.com/2020/04/23/us-unemployment-rate-numbers-claims-this-week-total-job-losses-april-23-2020-benefits-claims/. ↑
8. Meagan Wilson, Martin Kurzweil, and Rayane Alamuddin, “Mapping the Wild West of Pre-Hire Assessment: A Landscape View of the Uncharted Technology-Facilitated Ecosystem,” Ithaka S+R, December 11, 2018, https://sr.ithaka.org/publications/mapping-the-wild-west-of-pre-hire-assessment/. ↑
9. Kyle Bowen, “Open Badges Anatomy (Updated),” Class Hack, 2013, https://classhack.com/post/45364649211/open-badge-anatomy-updated. ↑
10. “What’s an Open Badge?” OpenBadges, 2020, https://openbadges.org/get-started; “Open Badges 2.0,” IMS Global Learning Consortium, 2020, http://www.imsglobal.org/activity/digital-badges. ↑
11. “IMS Global Open Badges 2.0 Validator,” IMS Global Learning Consortium, https://openbadgesvalidator.imsglobal.org. ↑
12. Shayne Spaulding and Martha Johnson, “Realizing Employment Goals for Youth through Digital Badges: Lessons and Opportunities from Workforce Development,” Urban Institute, 2016, https://www.urban.org/research/publication/realizing-employment-goals-youth-through-digital-badges-lessons-and-opportunities-workforce-development-0. ↑
13. Center for Education and Workforce, “Making Youth Employment Work: Essential Elements for a Successful Strategy,” US Chamber of Commerce Foundation, 2015, https://www.uschamberfoundation.org/reports/making-youth-employment-work-essential-elements-successful-strategy. ↑
14. “Soft Skills to Pay the Bills: Mastering Soft Skills for Workplace Success,” US Department of Labor’s Office of Disability Employment Policy, https://youth.gov/feature-article/soft-skills-pay-bills. ↑
15. Michelle R. Weise et al., “Robot-Ready: Human + Skills For the Future of Work,” Strada Education Network, January 16, 2019, https://www.stradaeducation.org/report/robot-ready/. ↑
16. “A Third of Youth Surveyed Globally by UNICEF Say Their Education Is Not Preparing Them with the Skills to Get Jobs,” Generation Unlimited, March 9, 2020, https://www.generationunlimited.org/press-releases/PwC-UNICEF-join-forces-to-boost-youth-skills-worldwide. ↑
17. “Youth Disconnection in America,” Measure of America, http://www.measureofamerica.org/DY2017/. ↑
18. Kristen Lewis, “Making the Connection: Transportation and Youth Disconnection,” Measure of America, 2019, https://ssrc-static.s3.amazonaws.com/moa/Making percent20the percent20Connection.pdf. ↑
19. David Abromowitz, “Hardworking Young People Are Building Our Future. Don’t Leave Them out of COVID Relief,” Miami Herald, May 13, 2020, https://www.miamiherald.com/opinion/article242675081.html. ↑
20. Grace Noppert, “COVID-19 is Hitting Black and Poor Communities the Hardest, Underscoring Fault Lines in Access and Care for Those on Margins,” The Conversation, April 9, 2020, https://theconversation.com/covid-19-is-hitting-black-and-poor-communities-the-hardest-underscoring-fault-lines-in-access-and-care-for-those-on-margins-135615. ↑
21. Robert Lerman, “America Can Skill Up Unemployed Workers Today for Tomorrow’s Jobs, Urban Institute, March 26, 2020, https://www.urban.org/urban-wire/america-can-skill-unemployed-workers-today-tomorrows-jobs ↑
22. Laura Aka, “COVID-19: ‘A Time of Immense Uncertainty,’” WorkingNation, 2 April 2020, https://workingnation.com/covid-19-a-time-of-immense-uncertainty/. ↑
23. Innovation Center Charter, Southern New Hampshire University Internal Document. ↑
24. Michelle Weise, “Welcome to the Sandbox,” Next Generation Learning Challenges, September 23, 2015, https://www.nextgenlearning.org/articles/welcome-to-the-sandbox; “About US,” Sandbox Collaborative, 2018, http://www.sandboxcollaborative.org/about/. ↑
25. Interview with Brian Fleming, Vice President of Strategy and Innovation, December 3, 2019. ↑
26. Lindsay McKenzie, “Marketing for a Massive Online University,” Inside Higher Ed , October 8, 2019, https://www.insidehighered.com/news/2019/10/08/how-marketing-helped-southern-new-hampshire-university-make-it-big-online. ↑
27. “Competency Based Programs,” Southern New Hampshire University, 2020, https://www.snhu.edu/program-finder/cbe-2. ↑
28. Google.org Full Grant Proposal, Southern New Hampshire University Internal Document. ↑
29. Ibid. ↑
30. Interview with Faby Gagne, Executive Director of SNHULabs, December 3, 2019. ↑
31. Interview with Brian Fleming, Vice President of Strategy and Innovation, December 3, 2019. ↑
32. Storyline LLC Qualitative Interviews of Opportunity Youth, Southern New Hampshire University Internal Document. ↑
33. Entangled Solutions Market and Landscape Analysis, Southern New Hampshire University Internal Document ↑
34. Milestone 1_SNHU AAP, July 31, 2018, Southern New Hampshire University Internal Document. ↑
35. “About LRNG,” LRNG, https://snhu.lrng.org/about/. ↑
36. Interview with Wil Zemp, Executive Vice President of Educational and Business Transformation, and Faby Gagne, Executive Director of SNHULabs, December 3, 2019. ↑
37. Interview with Michael Moretti, Director, Program Strategy and Design, December 3, 2019. ↑
38. Meagan Wilson, Martin Kurzweil, Rayane Alamuddin, “Mapping the Wild West of Pre-Hire Assessment: A Landscape View of the Uncharted Technology-Facilitated Ecosystem,” Ithaka S+R, December 11, 2018, https://sr.ithaka.org/publications/mapping-the-wild-west-of-pre-hire-assessment/. ↑
39. Interview with Wil Zemp, Executive Vice President of Educational and Business Transformation, December 3, 2019. ↑
40. “Core Score – A Soft Skills Assessment,” Core Score, https://corescore.jobs/employer_learn_more. ↑
41. “Innovate+Educate,” Innovate+Educate, https://innovate-educate.org/. ↑
42. “Technical Report: Core Skills Entry-Level Assessment,” Core Score, February 1, 2017, https://corescore.jobs/assets/corescore/cs-entry-report-ab12b77dd168f230593a324b664e0abe5781d3b50d747a0459541290648be973.pdf; “National Retail Services Initiative Competency Model,” National Retail Services Initiative, September 2016, http://vioconsulting.com/sites/548bb7f28cdaa71fda000002/theme/others/NRSI_Competency_Model_2016.pdf. ↑
43. Interview with Jordan Suhr, Associate Producer, December 3, 2019. ↑
44. Interview with Seth Corrigan, Senior Director of Assessment and Game Based Learning, December 3, 2019. ↑
45. “VALUE,” Association of American Colleges & Universities, October 7, 2014, https://www.aacu.org/value. ↑
46. “Problem Solving VALUE Rubric,” Association of American Colleges & Universities, https://assessment.provost.wisc.edu/wp-content/uploads/sites/92/2017/02/ProblemSolving.pdf. ↑
47. Authentic Assessment Platform: Preliminary Results of Milestone 2 Testing, January 31, 2019, Southern New Hampshire University Internal Document. ↑
48. Interview with Seth Corrigan, Senior Director of Assessment and Game Based Learning, December 3, 2019. ↑
49. Authentic Assessment Platform: Preliminary Results of Milestone 2 Testing, January 31, 2019, Southern New Hampshire University Internal Document. ↑
50. Interview with Seth Corrigan, Senior Director of Assessment and Game Based Learning, December 3, 2019. ↑
51. SNHU Core Score Badge Criteria and Competencies, Southern New Hampshire University Internal Document. ↑
52. “Technical Report: Core Skills Entry-Level Assessment,” Core Score, February 1, 2017, https://corescore.jobs/assets/corescore/cs-entry-report-ab12b77dd168f230593a324b664e0abe5781d3b50d747a0459541290648be973.pdf. ↑
53. Meagan Wilson, Martin Kurzweil, Rayane Alamuddin, “Mapping the Wild West of Pre-Hire Assessment: A Landscape View of the Uncharted Technology-Facilitated Ecosystem,” Ithaka S+R, December 11, 2018, https://sr.ithaka.org/publications/mapping-the-wild-west-of-pre-hire-assessment/. ↑
54. Interview with Faby Gagne, Executive Director of SNHULabs, December 12, 2019. ↑
55. Interview with Girlie Delacruz, Chief Learning Officer, December 4, 2019. ↑
56. Interview with Renee Marongwe, Director of Ecosystem Strategy and Analysis, December 3, 2019. ↑
57. Interview with Brian Fleming, Vice President of Strategy and Innovation, December 3, 2019. ↑
58. Lindsay McKenzie, “Marketing for a Massive Online University,” Inside Higher Ed, October 8, 2019, https://www.insidehighered.com/news/2019/10/08/how-marketing-helped-southern-new-hampshire-university-make-it-big-online. ↑
59. Top of Form“Badge: Problem Solving Skills,” LRNG, https://www.lrng.org/badge-assertion/my-badge/b89fc5804e1f15b7c835a8c1aa55da74b2433aa39b4909d1518e5b9b0266022e. Top of Form ↑
60. IMS Global Open Badges 2.0 Validator,” IMS Global Learning Consortium, https://openbadgesvalidator.imsglobal.org/results. ↑
61. Interview with Faby Gagne, Executive Director of SNHULabs, December 12, 2019. ↑
62. December Webinar with Employer Partners, December 12, 2018, Southern New Hampshire University Internal Document. ↑
63. Interview with Faby Gagne, Executive Director of SNHULabs, December 12, 2019. ↑
64. FEATuring You October Pop Up Event Recap, November 7, 2019, Southern New Hampshire University Internal Document. ↑
65. Meagan Wilson, Observation of December 2019 Boston event, December 4, 2019. ↑
66. Melanie Plourde, “New Soft Skills Assessment Helps Connect Opportunity Youth to Work,” Southern New Hampshire University, May 22, 2019, https://www.snhu.edu/about-us/newsroom/2019/05/soft-skills-assessment. ↑
67. Interview with Owen DeFrancesco, Director of Paid Social, December 4, 2019. ↑
68. Andrew Perrin and Madhu Kumar, “About Three-in-Ten U.S. Adults Say They Are ‘Almost Constantly’ Online,” Pew Research Center, July 25, 2019, https://www.pewresearch.org/fact-tank/2019/07/25/americans-going-online-almost-constantly/. ↑
69. Interview with Owen DeFrancesco, Director of Paid Social, December 4, 2019. ↑
70. Interview with Keisha Sheedy, Innovation Analyst, December 4, 2019. ↑
71. Interview with Krystal Siemon, Graphic Designer, December 4, 2019. ↑
72. Ibid. ↑
73. Interview with Cat Flynn, Deputy Director of Learning Science, December 4, 2019. ↑
74. Interview with Matt Ashby, Motion Graphics Designer, December 4, 2019. ↑
75. Ibid. ↑
76. Interview with Faby Gagne, Executive Director of SNHULabs, December 12, 2019. ↑
77. “About Us,” Mosio, https://www.mosio.com/about-us/. ↑
78. Interview with Faby Gagne, Executive Director of SNHULabs, December 12, 2019. ↑
79. Ibid. ↑
80. This aligns to 2017 findings by Gallup, which ran experiments survey response rates across digital delivery methods and found a 12 percent response rate to text message surveys. Jenny Marlar, “Using Text Messaging to Reach Survey Respondents,” Gallup, November 1, 2017, https://news.gallup.com/opinion/methodology/221159/using-text-messaging-reach-survey-respondents.aspx. ↑
81. Interview with Faby Gagne, Executive Director of SNHULabs, December 12, 2019. ↑
82. Jenna Joo, Jeffrey J. Selingo, and Rayane Alamuddin, “Unlocking the Power of Collaboration: How to Develop a Successful Collaborative Network in and around Higher Education,” Ithaka S+R, October 17, 2019, https://sr.ithaka.org/publications/unlocking-the-power-of-collaboration/. ↑
83. Gavin Tierney, et al., “Youth Co-Design of Responsive Digital Badge Systems: Disrupting Hierarchy and Empowering Youth,” CoDesign (2019): 1-17, Taylor and Francis, https://www.tandfonline.com/doi/full/10.1080/15710882.2019.1654522 . ↑
84. Katie Davis and Eve Klein, “Investigating High School Students’ Perceptions of Digital Badges in Afterschool Learning,” Proceedings of the 33rd Annual Association for Computing Machinery Conference on Human Factors in Computing (2015): 4043-46, https://dl.acm.org/doi/10.1145/2702123.2702413. ↑
85. “About Us,” Credential Engine, 2018, https://credentialengine.org/about/. ↑
86. “Counting U.S. Postsecondary and Secondary Credentials – A 2019 Report, Credential Engine, September 2019, https://credentialengine.org/counting-credentials-2019-report/. ↑
87. Laura Aka, “COVID-19: A Time of Immense Uncertainty,” Working Nation, April 2, 2020, https://workingnation.com/covid-19-a-time-of-immense-uncertainty/. ↑
88. “Job Opportunities for Workers Displaced by COVID-19,” National Retail Federation, https://nrf.com/resources/job-opportunities-workers-displaced-covid-19. ↑
89. “NYEC Community Response Feedback to COVID-19,” National Youth Employment Coalition https://nyec.org/2020/03/community-response-covid-19/. ↑
90. “Digital Tool Kit,” EveryoneOn, https://www.everyoneon.org/digital-tool-kit. ↑