Scholars have long pointed out the inherent flaws and potential biases of student evaluation systems — and the fact that these scores can possibly unfairly determine whether faculty members receive tenure or a raise. While these long-used assessment metrics were created with the intention of improving instruction, many institutions are starting to recognize that these largely opinion-driven standards do little to inform administrators on how well a faculty member is performing, which can be negative for the integrity of the business model. 

That's why the University of Oregon decided to turn the evaluation system on its head, with administrators developing an evaluation system that would rely more heavily on qualitative measurement, rather than having students rate instructors with arbitrary numbers. 

The problem: Biased performance metrics based on opinions

Sanjay Srivastava, a psychology professor at the University of Oregon, was on the original task force working on the new evaluation system. He told Education Dive what the typical issue with the typical rating mechanism. 

"As a psychologists, other experts look at my work and form expert opinions. Of course, that’s open to bias, but if the institution cares, you can mitigate bias," he said. "With the typical quantitative system, the numbers have the appearance of being objective and precise, and so people latch on to them. Then the number is the thing you care about, and the expert judgment takes a backseat, and often times with people not taking time to ask about the validity of those judgments."

Where this becomes an issue, is if students perhaps use evaluation systems in a retaliatory manner. Sierra Dawson, associate vice provost for academic affairs at the University of Oregon, has been working extensively on this project. She said the traditional evaluation system can be a problem because "you can have a professor who is a very knowledgeable person who doesn’t construct a very good learning environment, or you can have one that's organized but is not putting together a very effective learning environment."

The lack of appropriate insight means a professor could persist in a classroom to the detriment of students' outcomes, if his or her teaching has not been measured appropriately. The University of Oregon's website on evaluations is pretty transparent about this, noting, "studies show that students’ unconscious and implicit biases generally cause women and minorities to have lower evaluation scores than their white male counterparts, and that the level of the course and other factors may also affect the results."