Presence | Blog | The Wonder of Rubrics, Part Two: How to Develop a Rubric in 5 Steps

You may find yourself hearing about or wanting to use rubrics, but not sure where to start.

It can be intimidating to wade into the waters of a new assessment method. Can you be successful and get useful data if you’re a beginner here? You may feel pressure (and doubt your ability to) create an instrument from scratch. Perhaps you worry your approach is your own wacky method and the style or language you utilize would only make sense to you.

Deep breath.

Don’t stop yourself before you even get started. In this blog, I’ll walk you through five reasonable steps to create a rubric. I’m not promising it’ll be a snap, but hopefully, you’ll feel your anxiety or worries diminish as you read on.  

Before we get started, a few disclaimers:

• This is part two of a three-part series about rubrics. Check out part one for an overview defining and describing benefits of rubrics.
• There can be different ways to create rubrics, just as there are the multiple differentiated articulations of an assessment cycle all accomplishing similar results (each linked assessment cycle example pulled from NILOA’s featured assessment websites). In that spirit, this blog will look to cover common elements in rubric creation.
• For simplicity of example, we are assuming rubrics are being used to measure student learning outcomes. Content will primarily speak to analytic rubric creation, but nearly all of what’s shared applies to holistic rubrics.

1. Identify and articulate your outcomes

To begin, you first need to identify and articulate your outcomes. If you don’t yet have learning outcomes articulated, look to do that. Be sure you know which learning outcomes are to be measured in this instance and with this rubric.

2. Determine how you’ll measure

Once you know what you need to measure, determine how you’ll measure it. You’re using a rubric, but are you going to be doing observations? Artifact analysis? Obtaining self-reporting or peer review from students? Knowing this can help shape rubric development, particularly the language that you use. Thinking through this detail can also prepare administrative information for any data collection resources you’ll need.

3. Survey what already exists

In looking to begin developing your rubric, first, survey what already exists. Be sure to check internally for applicable rubrics or select content, as well as externally from other universities and organizations (such as AAC&U VALUE Rubrics). There are a lot of rubric examples out there, so see if anything meets your needs or helps you get started.

4. Identify these three things

Whether you’re making an existing rubric your own, looking to build off of some content, or starting from scratch, there are three pieces to determine for your rubric.

Scale points

Are you measuring competency (e.g., Beginner – Advanced), frequency (e.g., Never – Always), or extent to which something is performed (e.g., Not Met – Met)? How many points are in your scale: two, three, four, five, or more? Reporting needs can determine if your scale has words, numeric values, or both. In thinking about a grid, the scale points determine how many – and labeling of – the columns.

Dimensions

What is being demonstrated? Consider your learning outcomes and what traits would be present. Your intention may also necessitate some dimensions given the format or delivery method (e.g., dimension for visual appeal, volume, grammar). In thinking about a grid, dimensions determine how many – and labeling of – the rows.

Descriptions

What does each dimension look like at each scale point? In thinking about a grid, the description is the content describing and differentiating each point where row meets column. You can go about articulating these in various ways. One way is to fill in based on demonstration in progressing from one end of the scale to the other, while another method is to determine the anchors (highest and lowest level demonstration) and then fill in the middle of the scale.

These elements are related to one another. While you can work through them independently, the number of scale points and dimensions determines how many descriptions are needed. Likewise, articulating the descriptions may prove problematic or help you realize you need to collapse or combine some dimensions.

Both analytic and holistic rubrics have all of these elements. The difference is analytic rubrics score each description uniquely (e.g., multiple rows to be scored), while holistic rubrics group descriptions and dimensions so that scale points produce one overall score (e.g., one comprehensive row to be scored).

5. Test it out

You should test your rubric before using it for data collection.

• Share your rubric with others (faculty, staff, students) to see if it makes sense to others. A reader should see the logical progression of dimensions as descriptions ascend scale points.
• Pilot your rubric. One should be able to objectively and prescriptively score with the rubric, and not have to use personal discretion.

In both cases, look to adjust or revisit language and structure accordingly based on feedback for improvement.

While rubrics are only made up of three elements, they can be time-consuming to create. It takes great attention to detail, conceptualization of learning, and intimate understanding of the intervention. I’ve found the more you work with rubrics, however, the easier and more efficient you become in the drafting process.

Editor’s note: This is part two of three in a blog series about rubrics. Be sure to check out part one and part three.