Bryan Caplan  

Change My View: A Data Opportunity?

PRINT
Warsh on Friedman versus Samue... Anthropology Versus Commodific...
EconLog reader Peter Hurley sent me some interesting information on testing the efficacy of debate strategies.  Reprinted with Peter's permission:

Hey Prof. Caplan,

I have been reading your recent posts on econlog about the lack of research on debating tactics, and I wanted to propose a potential research avenue that you or someone you know might be interested in. 

I moderate a forum on Reddit called "change my view" which is essentially a debate forum, but specifically focused on people who have a view but are open to changing it.  As a part of the forum, we have a protocol for people to note and reward changes in their views.  Essentially, if someone's post changes your view, you include a greek delta symbol in the comment, and a bot comes around and gives the person who changed your view some internet points. 

You can access the site here: https://www.reddit.com/r/changemyview

The upshot of all this is that we have a tabulation of thousands of debates, and thousands of posts within those debates which were effective at changing someone's view.  I don't know exactly how to methodologically turn that into a useful conclusion about what's effective, but I figure you or one of your students or colleagues might be interested in giving it a crack. 

Good luck in your future debating endeavors.  

Best,

Peter Hurley



[Bryan again]

Sample bias is the obvious problem.  People who frequent this reddit are almost certainly unusual human beings.  But I still like the idea.  If anyone makes a serious go of it, I will blog it.


Comments and Sharing






COMMENTS (11 to date)
Peter H writes:

I'm the one who emailed Bryan, on the question of the data quality, I'll point out the following possible issues and advantages:

Issues:

CMV is heavily moderated. To keep it as a good forum, we need to enforce rules about civility and open-mindedness. As such many things get removed on a discretionary basis, though there are rules and guidelines about it.

DeltaBot isn't perfect. It's been broken for a few weeks now, though one of the other mods who is more adept than me has been keeping a crawler active to track deltas in the interim.

DeltaBot doesn't allow all deltas, even when it's working. The original poster of a submission can't be awarded a delta by another user, and any delta must be accompanied by a certain number of characters (200 I think) to explain why the view was changed.

Advantages:

We actually get a lot of submissions from people who aren't CMV regulars. So while the people who do the view changing may be regulars, the people whose views are changed are often not.

The data on removed submissions and disallowed deltas aren't destroyed completely. A researcher who was given a level of privileged access could scrape them via a crawler or something.

Sieben writes:

Another obvious problem is that people hate failure, so they are less likely to attempt to convince people of stickly propositions.

Radical libertarianism is definitely stickly.

Peter H writes:

Sieben,

I've never seen the word "stickly" used before, so I can't say exactly, though we do have plenty of posts and arguments about some more radically libertarian ideas (as well as some more radically statist ideas - and some just plain weird ones).

We do seem to get a hugely disproportionate number of posts about gender and transgenderism for whatever reason.

Dustin writes:

3.5 million registered users, 164 million visitors (June 015), 731 million unique visitors (2013)

https://www.reddit.com/about/

http://thenextweb.com/insider/2013/12/31/reddits-2013-stats-56-billion-page-views-731-million-unique-visitors-40-million-posts/

'Unusual'? This is the definition of 'usual' IMO.

It would be more accurate to say "People who frequent this EconLog site are almost certainly unusual human beings."

Anon. writes:

Well the first order of business is categorizing the arguments into different types. It's boring, menial work... Sounds like something graduate students would do.

Kevin L writes:

Personally, I've heard arguments that changed my view on moderately important topics, only to have my views revert or change again. That is something that cannot be tracked in a debate forum where the delta responses are, I assume, posted shortly after first reading an argument.

Mark writes:

Perhaps it could serve as a useful test environment.

Grab a random group of test subjects, ask them their views on various topics, and how strongly they hold those views.

Then get them to post their views on change my view, and see what happens.

Some of the data that could be interesting:
- Which sorts of appeals do they respond to (length, emotional vs. data-based, writing style, etc)
- Which arguments do they ignore
- How many rounds of argument does it take for their view to be changed?
- Was their view changed?
- Did their view stay changed? (requires follow up later of course)


Of course, you could do all of this in a more controlled lab environment, but since arguments tend to be threaded, you'd need to have a lot of "pre-programmed" responses to cover all of the directions the subject might take it. In fact, you could use (or browse) CMV to collect some of those arguments to prep for a lab test.

Ricardo Cruz writes:

Cool stuff to apply some hard data mining techniques to. :)

Ben writes:

Of course, one might well end up with the depressing finding that demagoguery, ad hominem attacks, and outright lies are the best way to change minds. I would guess that politicians know what they're doing; politics is a highly competitive environment in which those who are persuasive win and those who aren't get weeded out mercilessly (ok, Donald Trump being an obvious exception, but his personal fortune makes him somewhat immune to "weeding" :->). So the state of politics today makes me think that the debate tactics that work the best are not likely to warm Bryan's heart. But then, it seemed to me that Bryan's post was not about what debate tactics are the most likely to change minds, but rather, what tactics are demanded by integrity and honor. Which is, of course, a big reason why I've always been a Bryan fan, and why he has changed my mind more than once. :->

db writes:

A question for Peter H:

How do you verify that a given person actually holds the view they are purportedly willing to have changed by argument? That is, how would one control for disingenuous folks who hold already an opposing view and falsely claim to have been convinced by reason the error of their previous ways in an attempt to influence a debate in their favor? Said another way, how do you detect false flags?

Peter H writes:

db,

Human moderation. Such posts are prohibited under the rules, and have been removed in the past. Any user can report a post for violating one of our rules, and the moderators can act on that report, either unilaterally or after a discussion among ourselves.

Generally, false flags aren't so common, though they do happen. It's usually not too hard to tell, since people who aren't false flagging won't usually capitulate totally to the first replies they get.

Much more common are soapboxing posts where people post with zero intent or openness to changing their mind. We probably yank two or three of those on an average day.

Comments for this entry have been closed
Return to top