Showing posts with label Research Surveys. Show all posts
Showing posts with label Research Surveys. Show all posts

Monday, August 18, 2014

Surveys Can Make People Go Extreme by Esther Inglis-Arkell

http://io9.com/surveys-can-make-people-go-extreme-1621840701



There are all kinds of reasons why people don't tell the truth when asked questions. Sometimes they suddenly turn into fanatics. They hate, or love, anything. Here's how you catch people when they go extreme, or when they try to just get along.
We already know that people deliberately lie when given surveys on sex and drugs, but they also lie when given surveys about the importance of flossing and whether people should smoke in shopping malls. The difference is, many people don't even know that they're lying. People are driven to exaggerate (or even invent) their likes and dislikes, and so when they're asked to score, from one to five, their support for an issue or agreement with a statement, they avoid the middle and go right for one and for five.
This bias, called "extreme response bias" has annoyed many manufacturers, or politicians, who believed their targeted audience was passionately in favor of a new flavor of coke or a ban on littering, trotted the idea out, and gotten a lackluster response. Sometimes people are actually passionate about a subject, and sometimes they just want to be that way. Researchers took a look at separating out the two. They came up with a few guidelines to tell if people were inflating their opinions.
First of all, the more options you give a person, the more likely they are to go for the fringe opinion. A survey asking people to rate their experience on a scale of one to five will get far fewer extreme responses than a survey that asks people to rate their experience on a scale from one to ten. Individually, people with more education tend to be less extreme in their responses. The most telling variable, though, is another kind of bias.
Acquiescence bias is the tendency of a surveyed individual to go along with whatever the surveyor suggests. This is why researchers agonize over trying to make each question as neutral as possible. Ask people "don't you think smoking should be completely banned in malls," and they will tend to say yes. Ask them, "don't you think people should be allowed to smoke in public malls," and they will also tend to say yes. In order to be accurate, researchers can't tip their hands and let people know what answer they expect, or want. If, on the other hand, what the researchers want is to tell how many people responding to their questions are just going along with it, they can put out two surveys, one with a question that tips people one way, and one with a re-worded version of the question that tips people the other way.
Acquiescence bias tends to be a harbinger of extreme response. If people aren't going to be honest - either with the surveyors or themselves - they're at least going to be enthusiastically dishonest. So the more acquiescence everyone gets, the more extremity they should expect to see.

Friday, January 10, 2014

Leading Questions - Yes Prime Minister




Bernard Woolley: He's going to say something new and radical in the broadcast.

Sir Humphrey: What, that silly Grand Design? Bernard, that was precisely what you had to avoid! How did this come about, I shall need a very good explanation.

Bernard Woolley: Well, he's very keen on it.

Sir Humphrey: What's that got to do with it? Things don't happen just because Prime Ministers are very keen on them! Neville Chamberlain was very keen on peace.

Bernard Woolley: He thinks ... he thinks it’s a vote winner.

Sir Humphrey: Ah, that’s more serious. Sit down. What makes him think that?

Bernard Woolley: Well the party have had an opinion poll done and it seems all the voters are in favour of bringing back National Service.

Sir Humphrey: Well have another opinion poll done to show that they’re against bringing back National Service.

Bernard Woolley: They can’t be for and against

Sir Humphrey: Oh, of course they can Bernard! Have you ever been surveyed?

Bernard Woolley: Yes, well not me actually, my house … Oh I see what you mean

Sir Humphrey: You know what happens: nice young lady comes up to you. Obviously you want to create a good impression, you don’t want to look a fool, do you?

Bernard Woolley: No

Sir Humphrey: So she starts asking you some questions: Mr. Woolley, are you worried about the number of young people without jobs?

Bernard Woolley: Yes

Sir Humphrey Appleby: Are you worried about the rise in crime among teenagers?

Bernard Woolley: Yes.

Sir Humphrey Appleby: Do you think there is lack of discipline in our Comprehensive Schools?

Bernard Woolley: Yes.

Sir Humphrey Appleby: Do you think young people welcome some authority and leadership in their lives?

Bernard Woolley: Yes.

Sir Humphrey Appleby: Do you think they respond to a challenge?

Bernard Woolley: Yes.

Sir Humphrey Appleby: Would you be in favour of reintroducing National Service?

Bernard Woolley: Oh, well I suppose I might.

Sir Humphrey Appleby: Yes or no?

Bernard Woolley: Yes.

Sir Humphrey: Of course you would, Bernard. After all you told you can’t say no to that. So they don’t mention the first five questions and they publish the last one.
Bernard Woolley: Is that really what they do?
Sir Humphrey: Well, not the reputable ones, no, but there aren’t many of those. So alternatively the young lady can get the opposite result.
Bernard Woolley: How?

Sir Humphrey Appleby: Mr. Woolley, are you worried about the danger of war?

Bernard Woolley: Yes.

Sir Humphrey Appleby: Are you worried about the growth of armaments?

Bernard Woolley: Yes.

Sir Humphrey Appleby: Do you think there's a danger in giving young people guns and teaching them how to kill?

Bernard Woolley: Yes.

Sir Humphrey Appleby: Do you think it's wrong to force people to take arms against their will?

Bernard Woolley: Yes.

Sir Humphrey Appleby: Would you oppose the reintroduction of National Service?

Bernard Woolley: Yes.

Sir Humphrey Appleby: There you are, you see, Bernard. The perfect balanced sample.

Thursday, November 10, 2011

The Literary Digest Survey


The Literary Digest is almost certainly best-remembered today for the circumstances surrounding its demise. It conducted a "straw poll" regarding the likely outcome of the 1936 presidential election. The poll showed that the Republican governor of Kansas, Alf Landon, would likely be the overwhelming winner. This seemed possible to some, as the Republicans had fared well in Maine, where the congressional and gubernatorial elections were then held in September - as opposed to the rest of the nation, where these elections were held in November along with the presidential election, like today. This seemed especially likely in light of the conventional wisdom, "As Maine goes, so goes the nation", a truism coined because Maine was regarded as a "bellwether" state which usually supported the winning candidate's party.

http://en.wikipedia.org/wiki/The_Literary_Digest
http://historymatters.gmu.edu/d/5168/

http://faculty.muhs.edu/kresovic/MT177/assignments/05/Squire.pdf