 |
|
Liberal Media Bias - Blog Day Afternoon
Posted by Jeff (Friday September 19 2003 @ 02:05PM EDT)
|
|
"For months leading up this year's war on Iraq, the Bush administration implied that Saddam Hussein had a hand in the Sept. 11, 2001, attacks. The argument was well-received by Americans, and might have been the single leading factor behind public support for the U.S. invasion of Iraq. An oft-cited poll conducted by The Washington Post last month revealed that 69% of Americans continue to believe it likely that Hussein was personally involved in 9/11."
Read on...
|
|
|
|
By Tom (Friday September 19 2003 @ 04:28PM EDT)
|
Does anyone ever look at the data? They question is not designed to get at how well informed people are -- it's premised as "given your impression of Hussein, how likely do you think the following actions are." Let's just say that Hussein has had higher negatives than Gray Davis for the last 15 years. I'm suprised they didn't ask about the likelihood of drug use and pedophilia.
Poll Data
These are phone interviews. Phone interviews have enormous sampling challenges. One can only imagine with whom they actually speak.
Second, the question is weak. It's not "do you believe evidence has been found of Hussein's involvement?" -- It's "how likely is is that Hussein was personally involved?" And respondents can answer "very likely", "somewhat likely" "not very likely" or "not at all likely." The 69% we've seen repeated ad nauseum is the sum of those who answered "very likely" and "somewhat likely" that Hussein was involved. Essentially people are asked "given what you know about this guy -- do you believe it's likely he was involved?"
Of note is that this high response began showing up 2 days after September 11th, only then it was 78% and has since come down.
I conclude that this tells us nothing other than that many people believe Hussein to be a bad man and bad men can be imagined to do bad things. That the administration used this impression to further their goals is certainly true. I'm not sure it tells us anyting about the American public or the media -- other than that the media asks really dumb polling questions.
|
|
[ reply | parent ]
|
|
|
By Jeff (Friday September 19 2003 @ 07:05PM EDT)
|
The data and your spin were equally illuminating. It says what it says with a sampling error of +/- 3 points.
|
|
[ reply | parent ]
|
|
|
|
By Tom (Monday September 22 2003 @ 09:52AM EDT)
|
I try to do most of my spinning on hills. Clarifying terms in order to avoid over-interpreting data is something different.
I think an explanation of 'sampling margin of error' is in order. It is not a standalone guide to the quality of the survey. In any of these polls there are sampling errors and non-sampling errors. Sampling error, in this case +/- 3% is almost entirely derived from the size of the sample (not content) and assumes that other aspects of the survey were correct.
| Sample size | Approximate
Margin of Sampling Error |
|---|
| 100,000 | 0.3 |
| 10,000 | 1 | | 5,000 | 1 |
| 3,000 | 2 | | 2,000 | 2 |
| 1,500 | 3 | | 1,000 | 3 |
| 750 | 4 | | 500 | 4 |
| 400 | 5 | | 300 | 6 |
| 200 | 7 | | 100 | 10 |
The way to read the margin of error: "If we have conducted a scientifically sampled random survey of 1000 respondents, we would expect an error of no more than 3% -- because of the sample size." That's a big if and can only be answered by proxy based on the methodology of the surveyor. Even if they use well-considered methods (and I think the Washington Post does) they have the issue of response rate.
A typical media poll taken over a 3 day period will have a response rate of 25% on average. This means that for the 1000 people they talk to, 3000 either did not answer or refused to answer. This is another avenue for bias although recent studies I reviewed indicate that all other qualities being equal the different between high and low response rates is smaller than expected.
So now we get to my specific issue in this poll -- the question. In the Hussein - Al Quaeda example, the question asks the respondent to estimate the likelihood of an occurence or situation. Studies have shown that people are not good at likelihood or probability estimation. If instead you ask people do you believe or do you have evidence for believing, I contend you will get different results. Fortunately, PIPA has released a recent survey done the week of Sept 12th which addresses the issue with different questions. The full study is available here. I've tabled the relevant questions below.
|
|
[ reply | parent ]
|
|
|
|
By Matt (Monday September 22 2003 @ 10:28AM EDT)
|
People tend to answer polls based on how they think they should answer (or based on what they think the pollster wants to hear), rather than how they may actually feel.
|
|
[ reply | parent ]
|
|
|
|
By Jeff (Monday September 22 2003 @ 11:26AM EDT)
|
If nothing else, I've finally discovered why Matt answers my questions the way that he does...
|
|
[ reply | parent ]
|
|
|
By kevin the one-armed boy (Friday September 19 2003 @ 06:26PM EDT)
|
start sarcasm
The Washington Post would NEVER conduct a poll with leading questions that would attempt show the American public in a bad light.
end sarcasm
|
|
[ reply | parent ]
|
|
|
By Tom (Monday September 22 2003 @ 10:02AM EDT)
|
I don't think it's a leading question -- I think it's been over-interpreted by many. Asking about likelihood on Sept 13, 2001 seems appropriate and once you ask a question once, there is a desire to accumulate a time series on it.
|
|
[ reply | parent ]
|
|
|
|
|
Enlighten me, Marge
|
The most formidable weapon against errors of any kind is reason.
-- Thomas Paine
|
|
We Did Our Job!
|
|
|
