9 Data on Grants and Program Outcomes

Learning Outcomes

By the end of this chapter, learners will

• Understand the kinds of data on grant programs that can be captured
• Reflect on their goals for collecting data on grant programs and determine appropriate metrics that achieve these goals
• Develop strategies for data collection

Introduction

Many schools have launched grant programs to incentivize and support adoption, adaptation, or creation of open educational resources. The data that you collect about these programs will be largely context-dependent; it will depend upon how the program was structured, what you funded, and what you need the data for. For example, is the data intended for program improvement? Is it needed to justify future funding of your grant program? Is it to help you celebrate and showcase your great work? Consider your goals for data collection as you consider the many types of data and metrics that you could collect (which are detailed below).

Types of Data and Metrics to Collect

Recordkeeping

The following can be tracked by you or your team for each grant cycle, and recorded at the point when you are evaluating applications:

• Total amount of funds awarded
• Number of grants awarded
  • Amount of money awarded per grant
  • Type of grant awarded (ie., for adoption, adaptation, creation)
  • Title/program of grantee
  • Course(s) the project is anticipated to be used in
• Number of applicants
  • Type of grant applied to (ie., for adoption, adaptation, creation)
  • Title/program of applicant

Program Outcomes

The following information will need to be provided by grantees, and can be recorded by you as grantees share updates about their projects (either in conversation, email, or a formal report that asks for this information to be reported on):

• OER grant project formats (ie., open textbooks, multimedia, modules)
• OER grant project licenses
• Open platforms used to create and disseminate OER grant projects

Program Impact

On Teaching

Data about a grant project’s impact on teaching and course outcomes will need to be reported by the grantees themselves. You may wish to have grantees simply report based on their perception, based on anecdotes and interactions with students, or based on a formal research study within their course. If grantees will be required to do the latter and report back, then this requirement should be made clear at the time of application, and sufficient time should be provided for them to conduct and complete the study. Grantees may then benefit from some of the materials and exemplars provided elsewhere in this toolkit to guide their research design and process. Comparative data can be particularly helpful in demonstrating the efficacy of a grant. Potential datapoints of interest include:

• Number of students enrolled in course(s) where an OER grant project was used
  • Total estimated student savings on course materials
• Student engagement with or experience using OER grant project
• Effect on student course outcomes (perceived or evaluated)
• Student satisfaction with course and resource
• Any changes in how grantees approached their teaching

An OER project funded by your grant may go on to be used by a grantee in many courses over many years. You’ll want to decide for how many years you will track the related impact, and how often you will follow up with a grantee.

On Grantee Knowledge and Capacity around OER

Depending on the goals you set out to achieve with your grant program, you may wish to gauge whether grantees have increased their knowledge about OER throughout the program. Again, this is information that will need to be provided by grantees, either in informal conversation or through a formal report. You may want to ask about and learn:

• What they know about OER that they didn’t know about previously
• How they plan to share what they have learned with others
• Whether they feel equipped to embark on future Open Education projects

Program Utility

You may want to gauge whether grantees:

• Had enough funding to achieve project goals
• Had access to the expertise they needed to achieve project goals
• Had enough time to achieve project goals
• Benefited from the program structure and resources

Conclusion

Gathering data on the outcomes of your OER grant program is important and necessary for justifying its continuity, ensuring future funding of the program, and for program development. It’s therefore important to be intentional about the data that you collect. Carefully consider what it is that you want and need to know about the program, and ensure that the metrics you collect align with those intentions.

Resources

• University of British Columbia 2022 OER Fund Evaluation Snapshot

References

Gallant, J. (2022). 21. Data Collection and Strategies for OER Programs. In A. K. Elder, S. Buck, J. Gallant, M. Seiferle-Valencia, & A. Ashok, The OER Starter Kit for Program Managers. Rebus Community. https://press.rebus.community/oerstarterkitpm/chapter/chapter-21-data-collection-and-strategies-for-oer-programs/