Developing Online Assessments of Student Learning in a Hurry

Posted by Stacey Johnson on Wednesday, March 25, 2020 in Uncategorized.

by CFT assistant director, Stacey M. Johnson

Faculty, are you thinking about how to develop online assessments of student learning in a hurry? At Vanderbilt most courses meet face-to-face, but with campus closed through the end of the spring 2020 semester, we are all rethinking assessments. The CFT has resources to help. 

This blog post is Part II in a series about how to move your course online in a hurry. For Part I about rethinking face-to-face class meetings in an online environment, click here.

Even if you think online teaching seems like a manageable proposition, online assessment is a much bigger puzzle. Vanderbilt faculty thinking through that puzzle may want to spend some time getting to know the assessment tools we have available and some common ways instructors bend those tools to fit different teaching contexts. All of the tools in this post are already available to all faculty through Brightspace, our course management system. For help using Brightspace, reach out the our support team or visit our on-demand resources page.

As with the teaching options described in the first post in this series, the most important thing to consider is how to clearly communicate with students about expectations and requirements. Consider using the Brightspace Announcements tool for whole-class communications and one or more of the following tools for evaluating student work and providing useful feedback in a secure, FERPA-compliant environment. 

In this blog post, we will be using the framework established in the CFT’s 2019 guide, Assessing Student Learning by Michael Fischer and Joe Bandy, which is a great place to start as you think through your option in online assessment. 

Online and face-to-face examinations are often quite different. In a face-to-face classroom, exams might consist of a variety of questions types and exam strategies together in one test given on a single day. For example, a language course might include a test on vocabulary, grammar, listening, writing, and reading skills all in one exam. A lab course might test students on theoretical and practical skills all in one day during an in-class assessment. Another course might assess student recall, analysis, and writing skill in one paper test given during a class period. However, when conducting online assessment, we will often have to tease out the various components of our assessments in order to match the type of assessment with the best tool for the job. What level of test security is required for each test component? Should the assessment be synchronous or asynchronous, group or individual, iterative or one-time? How should feedback be communicated to students online? The answers to all of these questions will help to clarify which tools to use for the various parts of your overall assessment.

As you read this blog post, think about what different tools might best serve your purposes for each assessment strategy you plan to use. You might decide to rethink your assessment altogether in order to take full advantage of the affordances of online teaching, or you might opt to cobble together two or three different tools to create one comprehensive assessment that is consistent with your goals for the course. As always, once you know what your assessment will look like, Brightspace support at the CFT is here to help you make it happen using the tools available in Brightspace.

Essays/Student Writing

If you are having your students write for their assessment, there are several ways you might consider having them turn it that writing.

• Assignments

If you want students to submit their essays privately to you for review, the Assignments tool allows students to upload documents, record and submit videos, or add type, images, and/or links into a text box. The Assignments tool is one of the most versatile in Brightspace, and comes with an Annotation feature that allows instructors to add comments directly onto student papers without leaving the tool. This tool allows for multiple revisions of the same essay, TA grading, and you can even create groups and have students submit one assignment for the whole group. Check out these guides and resources related to Assignments.

The assignments tool is integrated with Turnitin which is an originality checker that looks for evidence of plagiarism. However, did you know that Turnitin also has two tools designed to make grading less arduous? Check out the Quickmarks and e-rater tools inside of Turnitin.

• Quizzes

The quiz tool in Brightspace is not just for quizzes! If you want to see what your students produce during a fixed amount of time without time to prepare and revise, the quiz tool allows instructors to set time restrictions and use the Respondus Lockdown Browser to prevent copying and pasting from documents or web sources. The question type called “Written Response” will give your students a text box in which to type their essay, and a quiz can consist of multiple questions and question types.

• Discussions

The Discussions tool facilitates asynchronous interaction. For example, if you want students to be able to read each other’s writing, Discussions is the space where asynchronous, collaborative work can happen among students. If you plan on breaking students into small groups for peer review, you would create the students groups, then set up a discussion space for each group along with clear instructions for how to post work and how to respond to each other’s work.

Exams and time-constrained, individual assessment

If you are considering how to translate your face-to-face exam into an online format, we strongly encourage you to start with the guide Giving Exams Online: Strategies and Tools by CFT associate director Dr. Cynthia Brame. This guide provides four clear options for giving tests online.

• Quizzes

The Quizzes tool in Brightspace is designed to make online testing possible. It includes a variety of settings to add restrictions to a test or quiz, includes questions types such as written response, multiple choice, True/False, and many others. For more detail on effectively using this tool, refer to the Giving Exams Online guide mentioned above. 

• Video Conferencing through Zoom or Virtual Classroom

One-on-one or small group oral examinations can be administered using Zoom or Virtual Classroom. This works particularly well for short, focused oral exams. To create a concrete record of the exam for your own review, you can use the recording function in your video conferencing tool. In Zoom, if you record to your own computer, students will not be able to access that recording unless you upload it and share it.

• TopHat

TopHat is Vanderbilt’s polling/classroom response system. It is integrated with Brightspace and has a range of options for secure online testing. Read the Giving Exams Online guide for more details on how to use this tool.

Tips for tests online: 

1. Best practices in online assessment often emphasize more synthesis questions that require students to produce original work and fewer recall questions that are easy to Google. Many faculty also lower the stakes of assessments online so they comprise a lower overall percentage of the final grade. Ongoing, generative work comprises a higher percentage of the grade.
2. There are plenty of ways to make your online test more secure, including the Respondus Lockdown Browser and Monitor. However, the higher tech the option, the more opportunities for the tech to fail. If you have no other option but to use a surveillance tool with your tests, try to minimize its use by applying it only to the essential parts of the test, and using a different tool for other parts of the assessment. Just because a test is done all in one hour in your face-to-face classroom does not mean it has to be so online.
3. Keep in mind that Vanderbilt relies on the student honor code as a mechanism for ensuring ethical testing. That does not change when we move online. Learn more here.

Presentations and Performances

• Pre-recorded Presentations

Students have access to the same video and audio recording tools in Brightspace as faculty. They can create webcam presentations, screencasts, or digital stories using tools they already have access to or using Kaltura Media, a Brightspace tool. Once they have a video presentation created, they can upload it to My Media and it will be available to share with you or with the whole class.

Brightspace also offers a Video Assignment tool that allows students to record themselves and share with the instructor. These are easily turned into peer reviewed assignments or graded by the instructor.

GoReact is a tool that is specifically designed to give give feedback on student-produced videos. The CFT’s Assistant Director of Digital Media, Rhett McDaniel, created this page with more information. 

If your students need help using these video creation tools, have them check out our student resources or reach out to brightspace@vanderbilt.edu 

• Live Performances with Interaction

Some performances require a live audience for participation. In these cases, there are two video conferencing tools integrated with Brightspace. With Zoom’s breakout room feature, you can break your class up into groups and multiple students can be giving performances at the same time to different groups while the instructor is able to pop in and monitor any group. 

Self-Assessment

• Surveys and Quizzes.

Surveys and Quizzes are very similar tools and both can be used for self-assessment. The primary differences between the two tools are that Surveys can be made anonymous and are not graded while Quizzes can be (but do not have to be) connected to a grade item in Grades. Either of these two tools can be used to walk students through a series of questions designed to help them analyze and assess their own performance through multiple choice, likert scale, or free response questions among other question types. 

• Reflective Assignments/Portfolios

If you have a good set of prompts for a reflective assignment or self-assessment, the assignments tool is great place for students to upload a variety of file types including documents, videos, and images, just like they might to an online portfolio. The instructor can give feedback using a variety of tools. 

For an example of reflective assessment in practice, check out this blog post on reflective final exams.

Peer Assessment

Many of the tools already discussed are great vehicles for peer assessment. Refer to this guide on Assessing Student Work when deciding how to organize your peer assessment, and start with a clear rubric or other tool that will help students focus their feedback and learn from others’ work. Rubrics can be easily built in Brightspace, and then deployed to any tool for instructor use. However, students do not have access to Brightspace rubric as an assessment tool. So, in order for students to use a rubric to assess their peers, they will need to have a copy of the rubric as Word doc or posted to Discussions, a Content module, or other common space.

Some of the Brightspace tools that can be effectively used to provide space for small groups of students to provide peer feedback include:

• Video Conferencing
• Discussion Forum
• Video Assignments
• Integrated tools like Peerceptive (for peer review specifically) and Perusall (for social reading, which can be adapted for collaborative feedback on written work)
• GoReact: Vanderbilt also recently acquired a campus license for a tool called GoReact that is specifically designed to give give feedback on student-produced videos, and it has a peer review setting to allow students to evaluate each other’s work. Find more information here. 

Assessment for Learning/Test-enhanced Learning

For many faculty members, a move to online teaching is an opportunity to review how and why we assess student work. You might consider adopting a model of assessment that emphasizes how students improve their performance over time, rather than weighting particular assessments so heavily that they become high-stakes, high-anxiety experiences. In a recent post, Thomas Tobin describes how these kinds of high stakes experiences increase the possibility that students will cheat or be overwhelmed with anxiety.

For useful background on the benefit of test-enhanced learning, check out this teaching guide by CFT Associate Director Cynthia Brame Test-enhanced learning: Using retrieval practice to help students learn. 

In Brightspace, there are tools that can help you provide students with this kind of low-stakes assessment and practice activities:

• Low-stakes or multiple-attempt quizzes

Quizzes in Brightspace can be set up as high-stakes exams, but they can also be set up to foster mastery. Using question pools, instructors can set up multiple practice quizzes that give students different sets of questions each time.  Quizzes can also be set up so that students have multiple attempts and only the highest attempt is entered in the grade book.

• Iterative written assignments

The Assignments tool in Brightspace can easily be set up so students can upload an assignment only once under a higher-stakes assessment model. However, it is also possible to allow student multiple submissions so that they can read feedback, revise their submission, and resubmit for a higher grade. By using the Assignments tool in Brightspace, all of a student’s submissions and all of the instructor’s feedback can be kept organized in one space.

Any tool in Brightspace can be used for test-enhanced learning, in fact. If you have an assignment that you would like to make lower-stakes, reach out to your CFT liaison. We are happy to consult with you about how you can make these kinds of changes to your courses.

Tips for Assessing Online and Keeping it Manageable

In case of any kind of emergency that may keep you or your students from campus, it’s important to remember that the tools you need to assess online work are already in place. For instructors developing online assessments in a hurry, remember to:

1) Manage your time by automating when possible. Online grading is notorious for taking much longer than you imagine. Take advantage of Rubrics, Turnitin Quickmarks, and other tools that help you pre-load the feedback you are likely to give to multiple students. Tools like Gradescope can be incredibly valuable for this purpose. When you need to give case-specific feedback, consider using the audio and video feedback tools if that will help you go a bit faster. Bonus: audio and video feedback can also feel much more personal and help strengthen your rapport with your students.

2) Use existing assessments. Your textbook may already have assessments built into their online platform. Your colleagues at Vanderbilt or other institutions may have developed online tools that they are willing to share. Look at the resources you already use and the people doing similar work and see how you can make use of what is already here.

3) Maximize your return on investment. Consider which tools you already know well and how you might leverage your existing knowledge as well as which tools will give you the most useful data on student performance. If you typically give a certain kind of midterm exam under normal circumstances, but the tools required to do the same kind of assessment online are overly challenging to set up in the short time you have available to you, it may or may not be a good investment to go ahead with your existing exam. You might decide that switching from a discrete item exam to a reflective paper makes more sense. Or you might decide that moving from performance assessment to an online quiz is the better solution. Collecting data about whether students are making progress is the goal of assessment, and it can happen many ways. Online teaching allows us to make some decisions about which data collection methods we choose and why.

4) Get help. Connect with colleagues who are faced with similar challenges or similar kinds of courses to crowdsource solutions. If you have questions about any Brightspace tools, check out our on-demand resources or reach out for support by phone or email. For help with classroom-based technologies or filming on-campus class sessions, email VUIT at av.support@vanderbilt.edu. You can also reach out to your CFT liaison for support as you think through assessment options.

What questions or ideas do you have for online assessment in your classes? Leave feedback and comments below!