Data on Need and Impact

CARL Open Education Working Group

10 Data on Need and Impact

Learning Outcome

By the end of the chapter, learners will:

Understand datapoints related to the need for and impact of OER

Introduction

Data on need can help support a funding or resource request for development or expansion of an OER program. For example, negative impact of textbook costs on students is a regular topic of inquiry – a 2023 South Carolina survey found that 82% of students did not purchase required materials and 39% took fewer courses due to materials costs. Need data can motivate administration to support more positive student outcomes and retention.

Data on impact, on the other hand, is essential for demonstrating the quantitative and qualitative value of OER programs and initiatives. Previous studies have found that OER improve course grades and decreases withdrawals, and save students millions of dollars. This data can help align OER initiatives with campus success metrics and demonstrate a positive return on investment.

COUP and SCOPE Frameworks

The Open Education Group proposed the use of the COUP Framework for evaluating the impact of OER:

Cost

The adoption of Open Educational Resources can impact a range of financial and cost metrics for students and institutions. Using these resources instead of traditional publisher textbooks or digital materials will save students money. There may also be other financial impacts, like changes in bookstore revenues and tuition revenues. The Cost strand provides empirical evidence about the magnitude and direction of the financial impacts of OER adoption:

Costs of textbooks previous assigned
OER support fee models
Changes in campus bookstore revenue
Changes in tuition revenue due to changes in drop rates
Changes in tuition revenue due to changes in enrollment intensity
Changes in tuition revenue due to changes in persistence
Changes in access to performance-based funding due to changes in drop, enrollment intensity, and persistence

Outcomes

OER increase student access to critical learning materials and expands instructors’ academic freedom, consequently improving student learning outcomes. The Outcomes strand provides empirical evidence about the magnitude and direction of the learning impacts of OER adoption:

Changes in the percentage of students receiving a C or better
Changes in rates of completion
Changes in drop rates
Changes in enrollment intensity
Changes in persistence
Changes in attainment of progress milestones (e.g., first 15 credits)
Changes in graduation rates

Usage

The permissions provided by open licenses allow students to use OER in a range of novel ways – for example, updating a history textbook based on recent events. Likewise, the permissions provided by open licenses allow teachers to engage in new pedagogical practices. The Usage strand provides empirical evidence about the ways instructors and students use OER and the degree to which impacts on learning outcomes vary with these uses. It operationalizes the idea of ‘exercising the permissions granted by open licenses’ by determining the degree to which students and instructors engage in activities described in the DIME model of OER adaptation:

Deleting material from the OER
Inserting other open material inside the OER
Moving material around within the OER
Editing material in the OER

Perceptions

What do instructors and students think about, and feel toward, Open Educational Resources? How do they judge their effectiveness relative to traditional textbooks? Their rigor and coverage? Do they find the formats, structures, and other design features easy to use? Frustrating? What about other stakeholders, like parents or policy makers – what are their thoughts and feelings toward OER? The Perceptions strand provides answers to these questions.

Adapted from The COUP Framework by the Open Education Group, licensed CC BY.

SCOPE Framework

The SCOPE Framework adds the component of social justice, which acknowledges the history of oppression and its impact on educational and social systems. Data in this category can include features of OERs that centre marginalized perspectives. For example, are images in OER representative of different population groups? Do instructors embrace liberatory practices? Do students feel included in course materials?

Basic Questions

How much does the average student spend on textbooks?
Did course completion rates or grades change when an OER was introduced?
How much did students not need to spend on textbooks because of the use of an OER?

More Complex Analyses

What is the impact of course material costs on educational equity? See for example the Virginia Course Materials Survey.
What is the return on investment of OER funding? This could consider aspects such as student savings as well as secondary impacts to institutional spending based on changes in student retention/success rates.

Resources

References

Clinton-Lisell, V. E., Roberts-Crews, J., & Gwozdz, L. (2023). SCOPE of Open Education: A New Framework for Research. The International Review of Research in Open and Distributed Learning, 24(4), 135–153. https://doi.org/10.19173/irrodl.v24i4.7356

Colvard, N. B., Watson, C. E., & Park, H. (2018). The Impact of Open Educational Resources on Various Student Success Metrics. International Journal of Teaching and Learning in Higher Education, 30(2), 262–276.

DawnDena. (n.d.). Financial Impact. Open Education Manitoba. Retrieved July 26, 2024, from https://openedmb.ca/about/financial-impact/

Open Education Group. (2023, March 1). The COUP Framework. https://web.archive.org/web/20230301022358/https://openedgroup.org/coup

Osterman, A. (2021). VIVA, Virginia’s Academic Library Consortium: VIVA: Virginia Course Materials Survey. https://vivalib.org/va/open/survey

PASCAL. (2023). Findings from the Textbook Costs Impact Survey. https://pascalsc.libguides.com/ld.php?content_id=73330989

License

Icon for the Creative Commons Attribution-ShareAlike 4.0 International License