10 Data on Need and Impact
Learning Outcome
By the end of the chapter, learners will:
- Understand datapoints related to the need for and impact of OER
Introduction
Data on need can help support a funding or resource request for development or expansion of an OER program. For example, negative impact of textbook costs on students is a regular topic of inquiry – a 2023 South Carolina survey found that 82% of students did not purchase required materials and 39% took fewer courses due to materials costs. Need data can motivate administration to support more positive student outcomes and retention.
Data on impact, on the other hand, is essential for demonstrating the quantitative and qualitative value of OER programs and initiatives. Previous studies have found that OER improve course grades and decreases withdrawals, and save students millions of dollars. This data can help align OER initiatives with campus success metrics and demonstrate a positive return on investment.
COUP and SCOPE Frameworks
The Open Education Group proposed the use of the COUP Framework for evaluating the impact of OER:
Cost
The adoption of Open Educational Resources can impact a range of financial and cost metrics for students and institutions. Using these resources instead of traditional publisher textbooks or digital materials will save students money. There may also be other financial impacts, like changes in bookstore revenues and tuition revenues. The Cost strand provides empirical evidence about the magnitude and direction of the financial impacts of OER adoption:
- Costs of textbooks previous assigned
- OER support fee models
- Changes in campus bookstore revenue
- Changes in tuition revenue due to changes in drop rates
- Changes in tuition revenue due to changes in enrollment intensity
- Changes in tuition revenue due to changes in persistence
- Changes in access to performance-based funding due to changes in drop, enrollment intensity, and persistence
Outcomes
OER increase student access to critical learning materials and expands instructors’ academic freedom, consequently improving student learning outcomes. The Outcomes strand provides empirical evidence about the magnitude and direction of the learning impacts of OER adoption:
- Changes in the percentage of students receiving a C or better
- Changes in rates of completion
- Changes in drop rates
- Changes in enrollment intensity
- Changes in persistence
- Changes in attainment of progress milestones (e.g., first 15 credits)
- Changes in graduation rates
Usage
The permissions provided by open licenses allow students to use OER in a range of novel ways – for example, updating a history textbook based on recent events. Likewise, the permissions provided by open licenses allow teachers to engage in new pedagogical practices. The Usage strand provides empirical evidence about the ways instructors and students use OER and the degree to which impacts on learning outcomes vary with these uses. It operationalizes the idea of ‘exercising the permissions granted by open licenses’ by determining the degree to which students and instructors engage in activities described in the DIME model of OER adaptation:
- Deleting material from the OER
- Inserting other open material inside the OER
- Moving material around within the OER
- Editing material in the OER
Perceptions
What do instructors and students think about, and feel toward, Open Educational Resources? How do they judge their effectiveness relative to traditional textbooks? Their rigor and coverage? Do they find the formats, structures, and other design features easy to use? Frustrating? What about other stakeholders, like parents or policy makers – what are their thoughts and feelings toward OER? The Perceptions strand provides answers to these questions.
Adapted from The COUP Framework by the Open Education Group, licensed CC BY.
SCOPE Framework
The SCOPE Framework adds the component of social justice, which acknowledges the history of oppression and its impact on educational and social systems. Data in this category can include features of OERs that centre marginalized perspectives. For example, are images in OER representative of different population groups? Do instructors embrace liberatory practices? Do students feel included in course materials?
Basic Questions
- How much does the average student spend on textbooks?
- Did course completion rates or grades change when an OER was introduced?
- How much did students not need to spend on textbooks because of the use of an OER?
More Complex Analyses
- What is the impact of course material costs on educational equity? See for example the Virginia Course Materials Survey.
- What is the return on investment of OER funding? This could consider aspects such as student savings as well as secondary impacts to institutional spending based on changes in student retention/success rates.
Resources
- University of Guelph student survey on textbook purchasing behaviours and outcomes
- Student Textbook and Course Materials Survey, Florida Virtual Campus
- Student Textbook and Course Material Survey, SUNY
- Student Surveys
- OER HUB Researcher Pack
- UBC Library Open Education Impact & Activity Report
- Calculating Student Savings
- A New Method for Estimating OER Savings
References
Clinton-Lisell, V. E., Roberts-Crews, J., & Gwozdz, L. (2023). SCOPE of Open Education: A New Framework for Research. The International Review of Research in Open and Distributed Learning, 24(4), 135–153. https://doi.org/10.19173/irrodl.v24i4.7356
Colvard, N. B., Watson, C. E., & Park, H. (2018). The Impact of Open Educational Resources on Various Student Success Metrics. International Journal of Teaching and Learning in Higher Education, 30(2), 262–276.
DawnDena. (n.d.). Financial Impact. Open Education Manitoba. Retrieved July 26, 2024, from https://openedmb.ca/about/financial-impact/
Open Education Group. (2023, March 1). The COUP Framework. https://web.archive.org/web/20230301022358/https://openedgroup.org/coup
Osterman, A. (2021). VIVA, Virginia’s Academic Library Consortium: VIVA: Virginia Course Materials Survey. https://vivalib.org/va/open/survey
PASCAL. (2023). Findings from the Textbook Costs Impact Survey. https://pascalsc.libguides.com/ld.php?content_id=73330989
A framework which looks at Cost, Outcomes, Use, and Perceptions (COUP) as it relates to the impact of OER or open pedagogy.