This is my 25th year in the business of polling, and now a new generation of beginners is telling me I do it wrong? I wish that someone had told me sooner. I might just have taken over my father’s grocery store. Come to think of it, so many of my colleagues have always told me I do it wrong, but there has always been a very powerful exception: no one else gets it more right than I have. Sure, I’ve blown a few here and there, but there’s a larger point to make: we are simply ahead of the curve, as I outlined in a recent interview with NPR.
I have recently taken criticism for my latest interactive survey of President Barack Obama’s job performance numbers, which show his positive rating at 49%, significantly lower than in other polls. While rushing to criticize the difference between mine and other polls, most fail to note that the findings on other polls are based on different scales. I use the excellent/good scale to measure positive job performance. I did not create this scale, but it is the same one I have always used. It had already been widely in use, I adopted it and I maintain it, because I think a four-point scale is more revealing than a simple approve/disapprove rating. In this scale, “fair” has always been placed in the negative column. I believe this gives a truer picture of those who truly give the president positive job performance marks. While “fair” can be interpreted in many ways, in my view ( and others who use this rating)it skews closer to the negative, especially when respondents are given a clear “excellent” or “good” option. In past years, my numbers for Bill Clinton usually showed him lower than polls that used a two-point approve/disapprove scale for job approval. The Democrats used to hate my polls in the 1990s. My numbers were consistently lower for George W. Bush for the same reason, and then it was the right’s turn to go bonkers.
Now here we are with President Obama – let’s face it, his numbers are lower. If they were still in the mid-60s he’d have a huge public opinion surge at his back, and like Clinton and like Bush, my lower numbers more accurately reflected this reality. You can’t move public opinion with those who just rate you as “fair”.
There are those who like to point to the online poll track record. I’m very proud of it. It’s different, and it’s vastly ahead of everyone else. On a national level, the numbers have been quite good. We have had the courage to release our numbers on a state level and frankly they need some work in some states. Don’t blindly judge the 2008 figures, because we stopped the online poll in mid-October – clearly a lot happens in a race in the final weeks and it’s hardly honest to judge the accuracy of a poll based on a survey taken weeks before Election Day. Why did we stop polling online weeks before the election, while we continued with our telephone polling? We simply had a very high volume of other interactive projects taking place at the time and made the decision internally to make those projects our focus.
The fact of the matter, however, is that our online polling results were quite good in 2004 and 2006, our online post- election poll in 2008 performed very well when compared with national exit polls. And our telephone sampling (including our highly praised state and national efforts this past election) makes us the pollster with the best overall record since 1996.
We will have subsequent white papers dealing in more detail about the online methodology, but be assured that panelists can’t just sign up and then think that they can “game” our poll. You sign up for a panel, and only occasionally are you invited to participate, because invitations are based on the oldest principle in this business, random probability sampling. Our critics have tried to make the case unlike in our interactive polling, every household has the same chance of being selected to participate in a telephone poll as another, but in fact those taking telephone polls are also “self-selected” by the very fact that they agree to participate and that phone polling response rates continue to decline. Coupled with the issue of the increasing number of households without a landline phone, we’re dealing with a narrower polling universe than ever before. We’re trying to restore legitimate probability methods to survey research, and it’s a discussable item. Let the legitimate discussion begin.
– John Zogby
Katy John Zogby, methodology John Zogby, methodology, Obama, polls, race, work, zogby
Loading ...