Bryan Caplan  

Improving Student Evaluations

Anti-Foreign Bias Strikes Agai... Income Distribution Facts...

When professors complain about grade inflation, they rarely mention that their students are the easiest graders of all. The main way that colleges evaluate professors' teaching is with student evaluations. Students typically rate how good their professor was on a 1-5 scale. In practice, most students give most professors the maximum score of 5 ("excellent"). This means that most of the variation in professors' evaluations comes from variation in a small number of disgruntled students.

For example, the typical professor at a typical school averages 4.4 out of 5. In a class with 20 students, if one student switches from a 5 to 1, that reduces your average by .2. All it takes to change a relatively good evaluation to a rather bad one is three hard-to-please students.

This gives professors perverse incentives to pander to the squeekiest wheels in the classroom, instead of enhancing students' average experience. A professor who wanted to maximize his evaluations would always give in to every student who came to his office to complain. They're the ones who make or break you come evaluation time.

I have a simple solution: stretch the scale upwards. If students call 60% of their professors "excellent," we need to add stronger adjectives to the list of responses. I suggest we add 6="best professor I've had this year" and 7="best professor I've ever had."

I still suspect students would overuse these options - during their four years, a student might give out ten 6's and five 7's, instead of four 6's and one 7 like they should. But my reform would publicly distinguish teachers who do their job and appease complainers from professors who change their students' lives but refuse to coddle them.

Comments and Sharing

COMMENTS (15 to date)
Nathan Whitehead writes:

Student evaluations of teachers are totally worthless. I speak from the "inside" here, from the point of view of mathematics. The only real use the evaluations are put to is direct feedback to the teacher in the comment sections, and to compare TAs teaching standard sections with exactly the same material and same grading policy.

We should go back to the "performer" model from ancient times. Students bring their daily tuition money, then the lecturer takes a break every once in a while and solicits donations for the day's teaching. Less attendance, less money for the teacher. Boring material, less money. Teaching is a service industry, but the service is absolutely terrible.

Pete M writes:

You could give students a "budget" of points to allocate to each of their professors. If a full-time student takes four classes in a sememster, let the student have, say, twelve points to allocate among all four of his or her teachers. Profs might not like this, because it would force them to compete against each other, but it would eliminate grade inflation.

If you want to add a kicker, allocate a pool of bonus money based on these points.

Of course, doing so gives the profs an incentive to inflate student grades, but a mandatory curve puts an end to that.

Will Baude writes:

Not to quibble, but shouldn't perfectly assessing and honest students to give more than one 7 throughout their four years? Assuming that all of them start as freshmen with no prior college experience, every first-year freshmen should give one 7 their first semester, because one of those professors will be "the best (they've) ever had". The next semester, if quality is randomly distributed across semesters, we'd expect half of those to give another 7. And third semester a third of them should give out a 7. And so on. The increase in knowledge and in class-bidding-weight might cause students to have better teachers in their later semesters which would increase the number of 7s per student per four years still further.

The only way around this would be either to let students retroactively *strip* a 7 ranking (which would be interesting, to say the least) or to ask students to rank 7 as "best professors I have had or will ever have" which is asking a little bit much in terms of foresight.

Paul N writes:

Another great idea that will never become popular. If it makes you feel better, I gave as many 1s as I gave 5s when I was a student...

James D. Miller writes:

To combat the problem you mentioned and grade inflation colleges should not count a student's evaluate if the student received a grade below B- in the professor’s class.

Ian Lewis writes:

I agree with Pete M. Why dont we evaluate professors the way we evaluate everything else: through supply and demand (and pricing). We should probably pay the most for the best teachers. This is how it works with piano and guitar teachers (and to a lesser degree, teachers in private schools). And when was the last time you heard of a music teacher shortage in the New York Times. Or that they were underpaid. I would imagine that the free-market would be the best system.

Robin W writes:

The problem is James M is there are some profs who do not give out many marks above B-'s, and they are often the ones students complain about most. Of course, a mandatory curve would get rid of that.

Timothy writes:

Oregon had scan-trons and long form short-essay evaluations. Of course, you didn't have to fill them out, but I ususally did. Signed ones went in the employee file, I signed every one I ever wrote. I think going to a short answer form instead of a 1-5 scale would be the best move.

Bryan Caplan writes:

Will Baude writes:

Not to quibble, but shouldn't perfectly assessing and honest students to give more than one 7 throughout their four years? Assuming that all of them start as freshmen with no prior college experience, every first-year freshmen should give one 7 their first semester, because one of those professors will be "the best (they've) ever had".


You're exactly right. The responses should be re-worded as 6="best professor I expect to have this year" and 7="best professor I've ever had." Then unless people make systematic mistakes, an honest student will on average give four 6's and one 7 during his undergraduate years.

Kevin Brancato writes:

I'd argue that, absent changes in the size of the scale or direct pay of professors, a system of evaluations can gain credibility (and perhaps utility) if a university-wide database of student evaluations were kept, with non-identifiable records (of both student evaluation and final grade) made public to the entire university, semi-identifiable information made to the professor being scored, and fully identifiable data attached to a student's confidential permanent record.

This would turn the system towards making sure that 1) students are held accountable for their use and misuse of the system, and that 2) professors have a way to fight a rogue student's negative ranking.

I'd argue that a professor should be able to compare any student's ranking of himself with the rankings the student has given to others, and how this correlates with the grades given by others to this student.

J L writes:

At the end of his post, Professor Caplan identifies one of the most salient and intractable problems for teachers at any level. In addition to making students "happy," teachers have a responsibility to impart (or, more accurately, to assist their students in the often difficult task of acquiring) knowledge. Teachers who teach well are not necessarily the most popular among their students. Subjective student evaluations may have some value, especially at the extremes, but colleges and universities rely much too heavily on such evaluations to determine whether someone is a "good" teacher.

There is some interesting and important research scheduled for publication next year demonstrating in multi-course mathematics courses (where "knowledge learned" can be reasonably identified) that there is no correlation whatsoever between subjective evaluations of teachers and what students learn in those teachers' classes.

I am generally in favor of market solutions to problems, but some of the proposals here do not identify all of the market forces that bear on the case. Teachers could (and, indeed, some have) become very popular merely by handing out high grades to every student in the course, but the value of the grade and ultimately the "education" would become less meaningful over time. That is why diploma mills generally cannot compete for long periods of time with more rigorous colleges with legitimate standards and more difficult grading policies.

David T. Beito writes:

We should never forget that administrators have always been the main force behind student evaluations. They regard these evaluations as an effective tool to be held in reserve to selectively punish or reward faculty. If a system uses numbers in any way, administrators can always think of new ways to manipulate it.

For this reason, I doubt that any solution (short of moving completely to discursive evaluations would have much value.

Will Baude writes:

Actually, I think 7 will have to be reworded to "= best professor I expect ever to have had (in college)".

David Hecht writes:

Well, it's an interesting concept. Too bad it's been tried and found wanting already! :-)

In the Navy, you are supposed to grade your subordinates using a scale whose top rating is "I consider this officer to be in the top 1%". Records show that approximately 40% of all officers receive this grade.

Steve writes:

Here's the REAL problem, as you progress through school a couple things can happen, and usually do happen.
1) your professors/teachers get better. I found my profs for upper level courses much better than for lower level courses, or at least their involvement and love of the subject being taught.
2) as a student you become more accutely aware of the ability of a professor.

My friends and I found that as we went through school, profs we thought were really bad, were in fact quite good, but did not care for the subject, or we did not understand their style of teaching, etc.

What would be best is to ask the student to review all their teachers at the end of their time at the institution, allowing them to fairly compare them all. OR ask them to only review their professors from the preceding quarter in comparision to only the other professors they had that quarter.

Comments for this entry have been closed
Return to top