Presence | Blog | How Student Affairs Professionals Can Assess Engagement in Virtual Programming

With campus disruptions caused by COVID-19, some people have questioned whether assessment is still necessary and relevant.

Through scrambling to adjust operations, staff are still driven to provide learning and support opportunities for students. In light of the uncertainty and the changes being made, I believe that now – more than ever – is a critically important time to assess our work. It’s the only way to know if we are delivering on our intended goals and outcomes.

 To be clear, I am not saying that you need to:

• Measure everything you previously planned to measure for the academic year
• Deploy the existing instruments you previously planned 
• Execute your assessment plans according to timelines set at the beginning of the year or semester

Plans you made last fall could look very different or seem impossible to maintain now. However, I would venture to say that your core purpose and intentions haven’t fundamentally changed. Moreover, the accommodations and adjustments you made raise real questions about engagement and effectiveness. 

I am also not suggesting that you should attempt to compare this year directly to past performances, nor to completely condemn or promote your adjusted operations. 

However, you’ll still want to determine whether your new intended or expected outcomes are met. Now is a time to stay true to the core of what assessment is, what it stands for, and what it asks of us. 

Assessment Fundamentals

When consulting with faculty and staff on assessment projects, I always pose these questions:

• How do you know students learn what’s intended?
• What does achievement of learning look like at your institution?
• What have you discovered about student learning that you can use to make improvements?

I let them know that assessment can answer these questions, which makes it relevant to everyone. 

The assessment process asks you to:

1. Articulate explicit learning outcomes (What do you hope students gain from the experience?)
2. Set appropriate criteria, standards, or targets for learning (What would successful student learning look like?)
3. Gather, analyze, and interpret evidence to determine how well performance matches expectations and standards (How are you going to collect and analyze your data?)
4. Use the results to document, explain, and improve performance (How are you going to use your data to inform changes?)

The first two elements are part of assessment planning, while the third involves executing on that plan, and the fourth is about acting on the results. 

We often have to navigate barriers and limitations in assessment planning. But I first encourage you to imagine the ideal assessment approach, before factoring in limitations on resources, dependencies, and feasibility. 

As the ideal becomes more practical, you can settle on a meaningful but manageable plan for the year. Institutional adjustments for COVID-19 can bring about unprecedented changes but if we rethink our assessment approach, we can still obtain insightful data.

Virtual Assessment

Although online operations engage students in different ways than what students may be used to, this fact doesn’t make assessment impossible. 

In fact, there isn’t a huge secret to remote or online assessment; all of the assessment fundamentals still apply.

I’m not trying to diminish concern or difficulty with adjusting approaches. But, it’s important to remember what is familiar, including that:

1. You still should have expectations for what learning occurs 
2. You still should set a target for how much learning should occur to be considered successful.
3. You are still able to collect data, analyze it, and interpret it.
4. You can still use your results to act upon improving your efforts.

For more advice and details on the above, see my past blog posts about articulating learning outcomes, prioritizing data needs, instrument selection/question creation, and taking action with results. Although they weren’t specifically written with online learning and engagement in mind, most of the elements still apply.

Here are some additional examples and opportunities that are unique to remote or online assessment in student affairs:

1. Virtual meetings or appointments

Any office that meets individually with students can still do so virtually. Since there are common topics covered and outcomes expected from such meetings, I often encourage the use of rubrics to capture demonstrated student learning. Some examples include: Goal articulation/achievement rubrics for academic advising, interview or resume review rubrics for career services, and rubrics to measure students’ understanding of degree requirements for enrollment. 

You might already be in the habit of taking notes during (or immediately after) your meetings with students anyway, so why not do so in a way that generates assessment data? You can utilize either digital forms or go with an old school paper-and-pencil-then-data-entry process.

2. Virtual events or educational programming

Three assessment methods come to my mind for virtual events. 

• Host a talk-back session with students – like an expanded, informal focus group – to hear what they gained from the experience.
• Ask students to complete a short reflection to a prompt, which you’ll later measure and quantify with a rubric.
• Ask students to self-report their knowledge gained or competency levels reached via a survey or rubric.

Methods two and three can be employed whether the events are held live (synchronously) or recorded (asynchronously).

3. Electronic reports/virtual award processes

You may have end-of-year reports or awards planned to recognize student leaders. These can still occur virtually or electronically. Materials can be submitted electronically, as can your reviews of reports or award submissions. There’s no need for pen and paper if you’re using electronic rubrics to score submissions. Plus, you can facilitate creative and memorable ways to virtually share and celebrate results!

 4. Online courses/packaged content

If you’re facilitating online courses, you have a host of digital engagement options. Quizzes, tests, papers, minute papers/short reflections, peer evaluations with rubrics, and surveys could all be used for data collection within a virtual environment.

The list could go on, but I don’t want to overwhelm anyone with details. I also don’t want to make this seem prescriptive or some kind of approved or endorsed list of the only options available. 

Remember: Prior to the COVID-19 outbreak, many institutions were already offering online/virtual student services as part of regular business. So, virtual or online assessment is not a new practice for our field, nor is it inherently inferior to traditional, in-person practices.

I hope this blog has been useful. While I’ve shared lots of details, examples, and suggestions, my message boils down to two points: First, remember your fundamental purpose – individually and for your functional area and its goals. Second, realize that assessment is still useful and helpful for answering important questions and advancing your practice. 

The path forward may be more familiar and comfortable for you than initially thought. Don’t let assessment be a stressor; let it be a supportive form of insight to help you make data-informed decisions that’ll best support student learning and development.

How have you assessed your virtual events? Connect with us on Twitter @themoderncampus and @JoeBooksLevy.

P.S. Looking for virtual event ideas? We have 53!