Arnold Kling  

Illustrated Primer Watch

PRINT
The "Buffett Rule's" Marginal ... In Person...

Tyler Cowen points to this article on a study of automated essay-grading.


"The results demonstrated that over all, automated essay scoring was capable of producing scores similar to human scores for extended-response writing items," the Akron researchers write, "with equal performance for both source-based and traditional writing genre."

File this under: The Diamond Age may be closer than you think.

As I have remarked recently, I think that the main capability that computers lack is the ability to interact conversationally with students, offering hints and encouragement as they try to learn. Solve that problem, and I believe you can automate education.


Comments and Sharing





COMMENTS (10 to date)
Lars P writes:

Remember that this is for grading essays written to be graded by a human.

If you let software do all the grading, people would learn how to write in ways that would not be liked by human readers, but would be scored high by the machine.

That's not to say that such software could not be very useful as a complement to the human judgement, both for the student and the overworked teacher.

Hugh writes:

And once education has been automated - what will be the use to human beings of that education?

It might have value as a hobby like, say, metalwork - but that's about all.

Saturos writes:

I've always believed my high-school English teachers could have been replaced with robots.

Roger Sweeny writes:

I suspect there is less here than meets the eye. State-wide and national tests of writing are graded by people who are required to use a "rubric"--basically a checklist of what should be included and not be included. Numbers are given for different parts of the rubric and then added up. It is a relatively mechanical process. The fact that it can now be done by machine is not terribly surprising.

The important question is whether the process tells you what you want to know. At present, there are complaints that it rewards paint-by-numbers writing that often doesn't show much understanding. Scoring by computer doesn't make it any worse, but it doesn't make it any better, either.

Steve Sailer writes:

Human grading of the SAT Writing essay is notoriously quick and dirty, so automating that job wouldn't be amazing.

john problem writes:

If you read English at British universities you get a chat or a lecture every now and then from your tutor. A computer could do that and provide a reading list and an essay request. Then universities would be cheaper and we could all enjoy higher education and get better jobs and live fuller lives and have lots of friends and drive shinier cars than the polloi - sounds good to me.

bob writes:

Computers won't take over teaching because of human nature. A big part of the incentive to learn is to prove yourself to your teacher. You don't care what a computer thinks of you. The quality of my chess falls almost involuntarily about 500 points if my opponent is a computer.

Brian Clendinen writes:

Roger Sweeny :"

The important question is whether the process tells you what you want to know. At present, there are complaints that it rewards paint-by-numbers writing that often doesn't show much understanding. Scoring by computer doesn't make it any worse, but it doesn't make it any better, either.
"

I totally agree with this based on experiance. I am a poor writer especilly in the grammar/spelling department but I have always had meat in what I wrote and knew how to structure my writing. I was able to get a 5.0 (which was 87%) on the GMAT which is graded by averaging both a Human and a computer score and only if the scores differ by more than .5 does another human look at it to over-rule the computer grading. There is no way at the time I was even in the top 50% in writing skills for business graduate students.

Testing grading on standered testing has a ways to go as far as I am concerned before it can closley messure a students writing skills as math/reading sections can.

Glen Smith writes:

Courses like English were the only courses I really needed discipline to study. Math and science I could easily see why I needed to learn the stuff and was often cool (especially learning how to blow stuff up). I really need a human teacher to make that English stuff come alive.

Bryan Willman writes:

And what part of education does this actually apply to?

English? As a primary language being taught to native speakers? Be still my beating heart...

Once again, a large part of "education" and even "human capital development" is around social processes - how to interact with people, how to behave in groups, how to avoid being bullied on the bus, how to compete for status, how to get laid, how to find a mate.

Whether a very costly tour at Harvard or GMU is a cost effective way to extend learning is a very legitimate question - but whether people need to learn them is not.

[And of course, in a society that demands signaling, there will be signaling. Whether it be a Harvard MBA or the right tatoos.]

Comments for this entry have been closed
Return to top