Putting the Courseware-in-Context (CWiC) Framework into Practice
Challenges of Implementing New Educational Technology at Scale
Earlier this year, Ithaka S+R was awarded a $2.46 million, multi-year grant from the Bill & Melinda Gates Foundation to develop, test, and scale new models of entry-level math instruction in higher education. Working closely with the project partners including TPSE Math (Transforming Post-Secondary Education in Mathematics), Acrobatiq, William E. Kirwan Center for Academic Innovation, the Urban Institute, as well as the two institutions that were part of a pre-pilot study during the 2016-17 academic year (University of Maryland, College Park and Montgomery College), Ithaka S+R is overseeing the overall development and implementation of an introductory statistics course based on Acrobatiq’s customizable adaptive-learning platform and courseware in seven additional pilot institutions in Maryland for Fall 2017.
One of the project deliverables will be a detailed “product specification” for the statistics course. A critical tool in developing this specification is the Courseware-in-Context (CWiC) framework, which was developed by a working group of Tyton Partners and Online Learning Consortium (OLC) with funding from the Gates Foundation. CWiC provides a common rubric against which practitioners (and providers) can evaluate the features of digital courseware, match it to their needs, and identify implementation steps. To assist others who may be considering to use CWiC (or evaluating courseware without knowing about the framework), I’ve written up a brief summary of the tool and how we’ve begun to apply it in the context of this project.
A Brief Overview of the CWiC Framework
Designed to support postsecondary decision-makers in navigating the dynamic market of digital learning solutions, the CWiC framework offers a research-based inventory of product capabilities as well as implementation guides that are critical for enhancing teaching and learning with the use of digital courseware. Here “digital courseware” is defined as:
…instructional content that is scoped and sequenced to support delivery of an entire course through purpose-built software. It includes assessment to inform personalization of instruction and is equipped for adoption across a range of institutional types and learning environments.
The framework is comprised of four main components:
- Product Taxonomy: an inventory of nine courseware product capabilities and 45 corresponding attributes (see Table 1) that are designed to aid the understanding of product functionality to support particular learning solutions (note: inventories of procurement and delivery platform capabilities are also available for supplemental resources)
- Research Collection: a list of peer-reviewed research tied to specific product capabilities/features in the Taxonomy to establish a connection between the courseware capabilities/features and efficacy research
- Course-Level Implementation Guide: a list of selected course-specific indicators (derived from OLC Online and Blended Learning Scorecards) to assess selected practices/policies that impact conditions for effective courseware implementation in a particular course
- Institutional-Level Implementation Guide: a list of selected institution-specific indicators (derived from OLC Online and Blended Learning Scorecards) to assess selected practices/policies that impact the conditions for effective courseware implementation in a particular institution.
Table 1. Functional capabilities in the CWiC framework and questions for exploration (adopted from the CWiC primer) [1]
| Functional capabilities | Description | Questions to ask for independent exploration and/or vendor conversations |
|---|---|---|
| Depth of Interaction | The presence of a variety of higher-order learning skills in instruction | How does the courseware product allow learners to demonstrate growth and proficiency, and does the variety of assessment tasks allow learners to show mastery of all learning outcomes in the course? |
| Measurement & Structure | The presence of academic structures and the capacity to assess learning in relation to them | How does the courseware product allow learners to demonstrate growth and proficiency, and does the variety of assessment tasks allow learners to show mastery of all learning outcomes in the course? |
| Scaffolding | Support structures to help learners achieve and grow beyond their current proficiency | How does the courseware product afford learners hints, alternate version or feedback when learners struggle with a task or ask for help? |
| Adaptivity | The adjustment of presentations of content in relation to knowledge of learners | How does this courseware product use adaptivity to deepen or accelerate learning? |
| Feedback | The deployment of reports, notifications, or visualization to learners or educators | How does the courseware product use or share its knowledge of learners and content in order to sustain engagement and encourage attainment? |
| Learner Autonomy | The ability for learners to impact or augment instruction based on their choices | To what degree do learners have the ability to impact or augment instruction based on their choices? |
| Collaboration | Collaboration is a requirement or opportunity for learners to engage with other people in the context of learning: peers, mentors, or educators | To what degree do learners need to cooperate, compete, or collaborate with other learners in order to ensure successful outcomes? |
| Customization Configuration | The ability for educators or course designers to alter learning or assessment content | How can educators or course designers change a courseware product so that it aligns with their course and program objectives? |
| Usability | Features of software and user-centered design that support sustained engagement | Are the features of software user-friendly and support sustained engagement of its users? |
These components are organized into three different instruments – the CWiC Primer, Designer, and Framework – to support the use of the framework for product exploration, comparing competing products, completing a full course or program review, respectively. One useful feature of the framework is that it allows for customization in the use of instruments to support different scenarios and goals for courseware adoption. For example, if the courseware adoption scenario involves “one course, many instructors” with a goal of “improving student success and engagement across sections,” the instructors and/or course designers can focus their CWiC review on priority capabilities – like measurement & structure, scaffolding, adaptivity, and collaboration – as they engage in independent exploration or conversation with a vendor.
Another valuable feature is that the CWiC framework is embedded in select products on the LearnPlatform and the EdSurge Product Index to support interactive exploration of various courseware products with communities of educators. Both LearnPlatform and EdSurge Product Index provide community-driven database of edtech product reviews. In the case of LearnPlatform, when educators sign up to join, they can connect with other verified educators in their institutions (or schools, school districts, agencies, organizations) to view others’ product libraries and reviews, request feedback on specific products, and share their own product insights. Such a community-based, interactive version of CWiC framework allows for developing a common language around the evaluation of emerging edtech products while situating them in particular contexts in which they will be adopted and used to meet particular needs.
Applying CWiC to the Maryland Project
What does all of this mean for Ithaka S+R’s Maryland project? The team is ramping up for the Fall 2017 pilot study, which will involve nine institutions that are geographically scattered throughout the state of Maryland, 26 instructors at different phases of their career, and over 1,000 students with diverse cultural, social, and educational backgrounds.
Certainly no one courseware will meet all learner needs in every possible scenario imaginable. In the case of Acrobatiq introductory statistics courseware, while it has robust functional capabilities in certain areas – like providing scaffolded and adaptive feedback to learners as they go through each module – it does lack features in other areas – like providing learners with opportunities to engage with peers, tutors, and/or instructors in the context of learning. In order to address this challenge, the project team has developed a hybrid instructional model in which the students will engage with the courseware prior to each class and the instructors will engage students in active learning through tailored instruction and exercises that are highly integrated with the courseware contents and assessments. Moreover, in an effort to support the pilot instructors’ work in this endeavor, the project team is organizing learning communities both at the local- and project-wide levels to provide ongoing professional development opportunities to enable the instructors to take full advantage of the courseware in their individual courses.
The CWiC framework provides a common language of evaluation for edtech consumers, so they can gain deeper understandings about their products’ capabilities while navigating a set of critical implementation considerations. It is also highly context-specific, leaving plenty of room open for product users to apply it to their own contexts-of-use. By applying this framework to the Maryland project, one of the few statewide coalition and scale-up efforts, we hope to contribute to the development of the CWiC framework in the context of new generation digital learning in entry-level college math courses.
[1] Source: http://coursewareincontext.org/cwic-wp/wp-content/uploads/2016/06/CWiC-Primer-Oct-2016.pdf