Author:
Aubree Evans
Subject:
Education
Material Type:
Teaching/Learning Strategy
Level:
Community College / Lower Division, College / Upper Division, Graduate / Professional, Career / Technical
Tags:
  • Scholarly Communication Notebook
  • License:
    Creative Commons Attribution Non-Commercial Share Alike
    Language:
    English

    BranchED Equity Rubric for OER Evaluation

    BranchED Equity Rubric for OER Evaluation

    Overview

    The BranchED Equity Rubric for OER Evaluation is intended for use by teacher educators to evaluation instructional materials for equity.

    Grunzke, R.Z., Jiles, T. Mayo, S., Grotewold, K., & Ianniello, P. (2021.) Equity Rubric for OER Evaluation. Branch Alliance for Educator Diversity.

    Introduction

    Branch Alliance for Educator Diversity (BranchED) is the only nonprofit organization in the country that prioritizes strengthening, growing, and amplifying the impact of educator preparation programs at minority serving institutions (MSIs), with the broader goals of both diversifying the teaching profession and intentionally championing educational equity for all students. BranchED’s stance is that “Quality is Diversity,” meaning that high-quality educator preparation embraces diversity, equity, and inclusion as fundamental (not tangential) to the preparation of all educators. We embody this stance through outreach, programming, and advocating for standards that center equity and inclusion. Central to this is ensuring representation of diversity within instructional materials so that teacher candidates see themselves in course materials. We believe that race-blind, culture-blind, language-blind, and economics-blind teacher preparation is obsolete. It does not adequately prepare teachers to engage with the diversity of the schoolchildren in their classrooms; diversity is as an asset and lever to enhance learning and the academic achievement of all students.

    Purpose

    BranchED developed the Equity Rubric for Open Educational Resources (OER) Evaluation to address the gap in instruments for evaluating OER teacher education instructional materials for equity. The rubric is intended for use by Teacher Educators (TEs) to evaluate OER with the goal of increasing the number of high-quality, equity oriented, and inclusive OER that represents communities of color.

    This tool has been designed to allow for the evaluation of resources at both the level of specific dimensions and also overall equity. Depending on the context, users of the rubric may want to compare resources based on total scores across all equity dimensions to select a more overall equitable resource over resources that may have a lower total score. In the creation of new instructional materials, a goal of a high overall equity score can map the design process so that it incorporates equity throughout. Another way to use the rubric is to focus on dimension subscores, which become especially useful when the rubric is used as a filtering tool. For example, one way the rubric can be used is if a faculty member decides, “I need to find a resource that is culturally sustaining, so I will look for an OER that scores high in the various criteria that make up the culturally sustaining dimension.” A resource that earns a high score in the culturally sustaining dimension may not get a high score in UDL, and, according to this example, that would be acceptable. 

    Instructions for Rubric Use

    The BranchED Equity Rubric for OER is designed to be used by educator preparation (EPP) faculty. It is organized around four broad dimensions of equity: Learner-Centered, Critical, Culturally Sustaining, and Universally Designed for Learning (UDL) which are color-coded within the document. 

    These four equity dimensions are broken down further into criteria, which are then measured through leveled indicators. The indicators are described using vocabulary specific to equity in education, the definitions of which are important to making a reliable and consistent assessment. For this reason, there is a linked glossary in Appendix A which includes the definitions to be used for the purpose of applying the rubric. The rubric measures four levels of evidence for each criterion ranging from Not Observed (0) to High (3).[1] Additionally, “look fors” offer examples of specific evidence to support the identification of each indicator. Screen tips for all look fors and glossary definitions may be accessed by hovering over their respective links in the rubric. Clicking on each link will take the user to the item in the full list of look fors or the glossary.

    After the criteria are evaluated for a dimension, an earned score (out of a total possible score) can be calculated and recorded for that dimension. Upon completion of this process for the fourth equity dimension, the user can add the four earned dimension-level scores to obtain an overall equity score for the resource. 

    We recommend that users take sufficient time to familiarize themselves with the rubric before employing it to evaluate resources. While an individual user can apply the rubric to the resources, we advocate that more than one rater from the same institution evaluate selected resources, obtain individual scores, confer to achieve consensus scores and then calculate the inter-rater reliability using Krippendorff’s alpha or another suitable measure of reliability for ordinal data, such as intra-class correlations (ICCs), Gwet’s AC2, or the Kendall rank correlation coefficient (also known as Kendall's τ coefficient).[1] More information about calculating Krippendorff’s alpha can be found here: https://www.statisticshowto.com/krippendorffs-alpha/.

    [1] Cohen’s kappa and its extension for more than two raters, Fleiss’s kappa, are designed for nominal (categorical) data, and therefore would not be appropriate for the ordinal data generated through the use of this rubric unless their weighted variants are used.

     

    [1] These levels of evidence correspond to traditional rubric performance levels in order to maintain the integrity of the rubric as a “pro asset-based’ evaluative instrument.