Surveys Can Make People Go Extreme by Esther Inglis-Arkell

http://io9.com/surveys-can-make-people-go-extreme-1621840701

There are all kinds of reasons why people don't tell the truth when asked questions. Sometimes they suddenly turn into fanatics. They hate, or love, anything. Here's how you catch people when they go extreme, or when they try to just get along.

We already know that people deliberately lie when given surveys on sex and drugs, but they also lie when given surveys about the importance of flossing and whether people should smoke in shopping malls. The difference is, many people don't even know that they're lying. People are driven to exaggerate (or even invent) their likes and dislikes, and so when they're asked to score, from one to five, their support for an issue or agreement with a statement, they avoid the middle and go right for one and for five.

This bias, called "extreme response bias" has annoyed many manufacturers, or politicians, who believed their targeted audience was passionately in favor of a new flavor of coke or a ban on littering, trotted the idea out, and gotten a lackluster response. Sometimes people are actually passionate about a subject, and sometimes they just want to be that way. Researchers took a look at separating out the two. They came up with a few guidelines to tell if people were inflating their opinions.

First of all, the more options you give a person, the more likely they are to go for the fringe opinion. A survey asking people to rate their experience on a scale of one to five will get far fewer extreme responses than a survey that asks people to rate their experience on a scale from one to ten. Individually, people with more education tend to be less extreme in their responses. The most telling variable, though, is another kind of bias.

Acquiescence bias is the tendency of a surveyed individual to go along with whatever the surveyor suggests. This is why researchers agonize over trying to make each question as neutral as possible. Ask people "don't you think smoking should be completely banned in malls," and they will tend to say yes. Ask them, "don't you think people should be allowed to smoke in public malls," and they will also tend to say yes. In order to be accurate, researchers can't tip their hands and let people know what answer they expect, or want. If, on the other hand, what the researchers want is to tell how many people responding to their questions are just going along with it, they can put out two surveys, one with a question that tips people one way, and one with a re-worded version of the question that tips people the other way.

Acquiescence bias tends to be a harbinger of extreme response. If people aren't going to be honest - either with the surveyors or themselves - they're at least going to be enthusiastically dishonest. So the more acquiescence everyone gets, the more extremity they should expect to see.