Jan’s Notes on Polling

Key Concepts in poll results

• Sample size or number of respondents to the poll
  • The number of people who took part in the poll.
  • Most polls ask questions of a random sample of people and then generalize the results to the group as a whole.
  • The larger the sample size the greater the chances of being able to make accurate generalizations.
  • Generalizations are not perfect.
• Confidence level
  • Pollsters don’t say that their findings are absolutely certain; they say that their results are probably accurate.
  • Pollsters use a mathematical formula to rate probability in the form of a percent, called the confidence level.
  • 100% confidence level is impossible so most pollsters use a 95% standard confidence level. That means being right 19 out of 20 times.
• Margin of error
  • Degree or range of accuracy. (If a confidence level shows the probability that a poll is accurate, the margin of error shows the degree or range of that accuracy.)
  • Pollsters try to keep the margin of error between 3-4%. One way to decrease the margin of error is to increase the sample size. Another way is to reduce the confidence level. These three statistics are interrelated.
• Bias in the questioning process
  • Questions must be phrased in a neutral way, but words are never value-free, nor do they always mean the same thing to different people.
  • Polls assume that people understand the questions completely and objectively, that they have well-informed opinions about the topic, and that they are answering the question honestly.
  • Reliability depends on:
    • How questions are asked.
    • How questions are set up by preceding questions.
    • Who is asking the questions and how.
    • When the questions are asked.
    • Whether the respondent has a considered opinion.
• Sponsorship of a poll
  • (EX: Cold cereal preferences paid for by Kellogg or laundry soap effectiveness paid for by P&G/Tide)
• Respondents’ quotes
  • Indicate that respondents agreed during poll questioning to talk to the media after the poll or survey was done. Such quotes are NOT people’s direct answers to the poll itself.

Political Polls

• Tracking Polls
  • “Horserace polls” are widespread during elections and critical in the final days before voting.
  • Useful for politicians but overused by media and importance to the public is uncertain.
  • Easy to do and inexpensive but they only accentuate the “who’s ahead” aspect of a race and not substantive issues.
  • Overcome tracking poll problems by analyzing voter shifts such as the effect of money or advertising, explain what else could sway public opinion in the last days of a campaign and interview public opinion experts.
• Exit Polls
  • Media care more about exit polls than campaigns do because they help analyze and report an election’s outcome.
  • Purpose of the exit poll is to understand the election, not to “call” it.
• Interesting?
  • Washington Post-Oct. 28, 2004
    • Post polling director Richard Morin says he listens as fellow pollsters talk nervously about the present and agonize about the future. “Cell phones, Caller ID and increasingly elaborate call screening technologies make it harder than ever to reach a random sample of Americans,” he writes. “Costs are soaring as cooperation rates remain at or near record lows. In some surveys, less than one in five calls produces a completed interview — raising doubts whether such polls accurately reflect the views of the public or merely report the opinions of stay-at-home Americans who are too bored, too infirm or too lonely to hang up.”

Source: Jan Leach