The Nature of the Interactive

Zogby International's online polling methodology, known as Zogby Interactive, conducts surveys via the Internet using a proprietary panel of respondents that now numbers several hundred thousand people. This is a methodology that has been 11 years in development, and every political cycle yields more encouraging results that prove that not only is Zogby's Internet survey research an accurate and reliable tool, but also it is going to be an important methodology of future survey research.

The nature of traditional live caller telephone polling, a methodology also used by Zogby, is changing as telephone technology changes and more American households use either digital telephones or cell phones - without a traditional landline telephone in their homes. Because of these changes, which reflect a more mobile society and technology that accommodates that mobility, public opinion research using the Internet as the communications device is an obvious solution. The Zogby Interactive survey methodology has become a leader in the industry.

Here is how it works:

• When an interactive survey is initiated, a random sample is drawn from this pool of potential respondents, and Zogby sends them an email invitation to participate in a survey, which includes a link that will take them to the survey on Zogby's own secure servers. Respondents never know when they are going to get an invitation to take part in a survey, and they never know what the subject matter included in the survey is going to be until they click on the invitation link in their email.
• Statistics from the recent experience of the Zogby interactive survey emphasizes the point that Zogby respondents are unable to covertly distort the results of an interactive survey. An analysis of the Zogby panel shows that, in a recent 12-month period, 61% of Zogby respondents had taken only one or two surveys (41% had taken just one survey, and 20% had taken a second). Zogby survey respondents do not "opt-in" to participate in polls, they are invited - much like telephone poll respondents are invited to take a survey over the phone by answering the phone and agreeing to answer questions. Members of the Zogby panel do, in fact, opt-in to the panel, however, with the understanding that they may be invited to participate in a survey from time to time.
• The email link inside the Zogby interactive poll invitation expires after one use, which is just one of many security measures Zogby has in place to guarantee the veracity of the polling methodology.
• The survey is active on Zogby's secure servers until such time as it is deemed that the proper number of respondents have completed the survey to fulfill the requirements of that particular polling project.
• When needed, Zogby's interactive polling is supplemented by telephone polling conducted in Zogby's on-site telephone call center in Upstate New York to guarantee a full and proper representation of all demographic groups.
• Once the desired number of respondents has completed a survey, Zogby technicians process the results, weighting each poll according to its standard practices to reflect the age, gender, geography, race, religion, political affiliation of the population a particular survey is designed to emulate. Other weights may also be applied, depending on the survey.

Using the online interactive panel has some unique advantages compared to traditional telephone polling that benefits from a modern technological society. Studies of the subject show the Internet access penetration in the United States today is over 80%, and that percentage continues to increase. This, combined with the fact that an increasing number of younger respondents do not own landline telephones, suggests that the Internet is becoming the most viable means of reaching some key demographic groups. From an operational point of view, interactive polls are attractive for another reason - their lower cost. They also share some of the desirable properties of focus-group studies because they allow for more in-depth questions, the ability to test subjects using audio, video, and other multi-media techniques. They also allow for more expressive and detailed open-ended responses. In addition, larger sample sizes allow for more powerful cluster analyses and prompt recognition of emerging groups and patterns.

Yet another significant benefit of the interactive survey methodology as employed by Zogby International is that a survey can be fielded in a very short amount of time, yielding over-night results in some cases with sufficiently large sample sizes to satisfy client demands and allow for satisfactory sub-group analysis. In a world where events sometimes change circumstances by the hour, not the day or week, this is an important factor that makes the industry more relevant to the world in which it exists.

Quality control

Zogby is careful about its quality control, and has put in place many measures to guarantee the veracity of the interactive polling methodology, and to keep its panel as pure as possible. It keeps an on-going record of data points on each respondent, so that, if any glaring inconsistencies develop over a period of multiple polls, that respondent can be eliminated from the pool.

In addition, Zogby follows closely the demographic trends in its interactive panel, and is able to do so because when panel members first sign up for the panel, they are asked a screen of demographic questions which are attached to their account. Panel members are not identified by name in any way to protect their anonymity, but other identifying data is collected and catalogued by Zogby to help it maintain and track its database.

As with any opinion research process, the demographics of the respondents are important, and Zogby closely tracks the makeup of its panel. When the panel shows the beginnings of a trend that indicates it may be headed toward a divergence with the demographic makeup of the U.S. population at large, adjustments are made to make sure that panel remains in sync with general population trends. One way that is achieved is through the recruitment of new panelists with particular demographic characteristics that keep the Zogby panel in balance.

It is important to note Zogby's aggressive outreach efforts to grow its panel. It is constantly working to expand its panel membership through the use of its own call center and by using various lists to reach out to potential respondents. Almost all of these people would not normally sign up for the Zogby panel had they not been contacted by Zogby and invited to do so. The Zogby panel is not formed solely through self-selection by potential panel members themselves. This is an important aspect of the company's effort to thwart the tiny percentage of people who would sign up for the Zogby panel in order to skew its opinion research efforts. Those people represent such an infinitesimal portion of the panel as to be statistically insignificant.

Another aspect of Zogby interactive quality control is to randomly double-check the veracity of individual responses to all online surveys by contacting up to 4% of each online respondent pool via telephone. In that telephone follow-up from Zogby's own on-site call center, live operators ask respondent a random sample of the survey's questions simply to check to see that their responses to the interactive survey were accurately and reliably recorded. No other pollster in the industry takes such a step to guarantee the security and veracity of their polling product.

Interactive Polling Horserace results

The accuracy of Zogby Interactive polling of key political races continues to improve with each election cycle. The races in the 2006 Congressional midterm elections and in the 2004 presidential election has validated the methodology, and improvements continue to be made. Zogby Interactive correctly predicted the winner in 17 out of 18 U.S. Senate races in 2006, a tumultuous political year to say the least. As is the case in all polling methodologies, Zogby's interactive polling of these races differed from the actual results - in a handful of cases by a significant percentage. This is not, however, an indication of weakness in the Zogby model, but rather is reflective of the nature of political polling of all methodologies - that the larger a margin that one candidate holds over another, the larger the gap between polling results and the actual election results can be. This "wobble" in political blow-outs is a simple fact of life and occurs because of a "band-wagon effect" that develops in cases where one candidate is unusually strong and the other is unusually weak. It may seem odd that pollsters might prefer closer political races to blowouts, but it is true - partly because voters tend to pay closer attention to the campaigns in close political races, and so have more informed opinions. The more informed the opinion of a respondent, the more reliably pollsters can capture that opinion.

In 2008, Zogby's interactive polling was equally impressive. However, it is important to note that, due to the high volume of interactive projects conducted by Zogby in the fall of 2008, our last interactive horserace poll of the 2008 Presidential election occurred on October 4, 2008. At that point, through to the election, Zogby continued to poll via telephone in a partnership with media clients. But because our interactive horserace polling stopped fully a month before the presidential election to accommodate the needs of non-political clients, it is silly if not willfully disingenuous to point to the numbers from the early October Zogby interactive poll and compare them with the final election results that occurred a full month later.

There are many reasons this apples-to-oranges comparison is inappropriate, among them the mere fact that nearly 8% in the interactive survey a month before the election said they were yet undecided. On Election Day, there were no more undecided voters - they either voted or they didn't. If you then factor in the margin of error, the Zogby interactive survey was as accurate as any poll in the industry.

A more appropriate comparison would be to compare our interactive results with a phone poll taken at the same time. Below are the results from the final Zogby Interactive horserace poll on October 4, 2008 and a Reuters/C-SPAN/Zogby telephone poll conducted between October 4 and October 6.

In addition, internal Zogby interactive polling conducted immediately after the election shows that our interactive results fell mostly within the margin of error of results from exit polls.

In summary, it is important to note that Zogby's industry-leading efforts to develop and perfect an accurate and reliable online polling technology has led it to the top of the field. It has also made the company's work a target for those who, for whatever reason, want to sit back and fire spitballs from the cheap seats. But the more polling experts understand about the Zogby method, the clearer it becomes that its efforts to build an interactive methodology that meets industry standards have been a success. Continuing refinements constantly improve the process and the results.

Most in the industry have seen the serious technological and societal changes and challenges facing the opinion research field and have shied away from exploring new ways to sample the ideas and thoughts of the American public - daring to communicate with people the way they were using new technology to communicate with the world around them. But Zogby has taken a different tack, boldly pushing forward, embracing technology while maintaining its fierce independence and polling political races that others would never risk. In doing so, the Zogby online polling product has made great strides forward and it now finds itself at the top of the pack. The tremendous popularity of the interactive poll, particularly in the business world, is proof that the ground-breaking efforts of the Zogby interactive poll is on track. The objective statistical evidence of its reliability and accuracy backs that up.

Zogby International's online polling methodology, known as Zogby Interactive, conducts surveys via the Internet using a proprietary panel of respondents that now numbers several hundred thousand people. This is a methodology that has been 11 years in development, and every political cycle yields more encouraging results that prove that not only is Zogby's Internet survey research an accurate and reliable tool, but also it is going to be an important methodology of future survey research.

The nature of traditional live caller telephone polling, a methodology also used by Zogby, is changing as telephone technology changes and more American households use either digital telephones or cell phones - without a traditional landline telephone in their homes. Because of these changes, which reflect a more mobile society and technology that accommodates that mobility, public opinion research using the Internet as the communications device is an obvious solution. The Zogby Interactive survey methodology has become a leader in the industry.