The Exit Poll Discrepancies- Still unexplained, ignored by the media
by : Steven F. Freeman, PhD Monday November 15, 2004 - 20:00
> Elections-Elected
> USA
3 comments
![]() Steven F. Freeman, PhD Author’s note: Given the timeliness of the subject matter, I have released this paper despite not having the opportunity to use normal academic safeguards. If you have any questions or comments, please write. Likewise, if you publish or post it to web pages, electronic bulletin boards, or other electronic archives, please let me know.
I have tried to be as rigorous as possible in my data collection, review, and analysis and I believe it compares favorably to the vast majority of commentary currently in the public domain. To hold it to an academic standard of rigor, however, requires extensive peer review; this work has barely begun to be challenged by - and improved from - this peer review process. I will revise the paper as legitimate objections are raised, as more data becomes available, and as I learn more. ![]() The media has largely ignored this discrepancy (although the Blogosphere has been abuzz), suggesting that the polls were flawed, within normal sampling error, or that it was a statistical anomaly. In this paper, I examine the validity of exit polls, the likelihood sampling error, the possibility of statistical anomaly, and other explanations thus far offered. 3 Those who purchased the information signed agreements prohibiting the
release of the data. (Martin Plissner, "In Defense of Exit Polls: You
just don’t know how to use them" Slate Thursday, Nov. 4, 2004) On Exit Polls Caveats aside, the data appears to be good.7 In general, we have every reason to believe that exit polls are accurate survey instruments. Exit polls are surveys taken of representative respondents from the overall voting population. Although exit polls have not been academically studied, both the logic behind them and experience suggest that we can use these surveys to predict overall results with very high degrees of certainty. It’s easy to get a statistically valid representative sample; and there is no problem with figuring out who is going to actually vote — or how they will vote. Thom Hartmann of CommonDreams relates that in his native Germany, ... people fill in hand-marked ballots, which are hand-counted by civil servants, watched over by volunteer representatives of the political parties. ... even though it takes a week or more to count the vote ... the German people know the election results the night the polls close because the news media’s exit polls,
for two generations, have never been more than a tenth of a percent off.8
Dick Morris, Republican consultant and Fox News regular, concurs: Exit polls are almost never wrong ...So reliable are the surveys that actually tap voters as they leave the polling places that they are used as guides to the relative honesty of elections in Third World countries. When I worked on Vicente Fox’s campaign in Mexico, for example, I was so fearful that the governing
PRI would steal the election that I had the campaign commission two U.S. firms to conduct exit polls to be released immediately after the polls closed to foreclose the possibility of finagling with the returns.9
5 Richard Morin, "New Woes Surface in Use of Estimates" Washington Post, Thursday, November 4, 2004; Page A29 6 I’m not suggesting conspiracy here I would hate to even imagine the volume of calls and emails they have had to manage in the past week just (defensively) noting that the data that I am using is the best available. 9 Dick Morris, "Those faulty exit polls were sabotage" The Hill Nov. 4, 2004 http://www.thehill.com/morris/110404.aspx ... results are very precise; In the 2003 Salt Lake County mayoral race, the KBYU/Utah Colleges Exit Poll predicted 53.8 percent of the vote for Rocky Anderson and 46.2 percent for Frank Pignanelli. In the actual vote,
Anderson carried 54 percent of the vote to Pignanelli’s 46 percent.
True to their word, predictions in this year’s contests were quite accurate. In the Utah presidential election, for example, they predicted Bush 70.8%, Kerry 26.5%. The actual was Bush 71.1%, Kerry 26.4%. Consistently accurate exit poll predictions from student volunteers, including in this presidential election, gives us good reason to presume valid data from the world’s most professional exit polling enterprise. Data and Statistical Analysis Three critical Battleground states The conventional wisdom going into the election was that three critical states would likely determine who would win the Presidential election — Ohio, Pennsylvania, and Florida. Sure enough, they did, with Bush winning two of three and ascending to electoral victory. In each of these states, however, exit polls differed significantly from recorded tallies. Data (Ohio). CNN reported the exit poll as illustrated in Figure 1. Combining the male and female vote, weighted for their percentage of the electorate (47% male), Kerry’s predicted share of the total Ohio vote was 52.1%12. Doing the same for Florida and Pennsylvania, and adding in final tallies (NY Times, Sunday evening), we derive Table 2. ![]() 10 Martin Plissner, Exit Polls to Protect the Vote, New York Times 10/17/04 11 http://exitpoll.byu.edu/2004results.asp. As far as I have been able to determine, this was the only other exit poll conducted on the 2004 presidential election. I had thought that Zogby also had an exit poll, but haven’t been able to verify this; they may have been using the same National Election Pool data, when they declared Kerry the winner in Ohio on Election Night. See William Douglas, "Early exit polls come under fire" The Mercury News (11/3/2004) ![]()
Statistical significance, which means that the discrepancy is such that it is unlikely to occur by chance, depends on three factors the size of the discrepancy, the sample size, and the level of significance (just how unlikely does it have to be?) For analysis purposes, we could choose any measure: Bush’s differential, Kerry’s differential or the differential between them; it all works out the same. Based on the analysis that will follow, I’m going to examine Kerry’s percentage of the vote. ![]() Figure 2. Statistical prediction of Kerry’s true percentage of the vote in Ohio The likelihood of any two of these statistical anomalies occurring together is on the order of one-in-a-million. The odds against all three occurring together are 250 million to one. As much as we can say in social science that something is impossible, it is impossible that the discrepancies between predicted and actual vote counts in the three critical battleground states of the 2004 election could have been due to chance or random error. 17 Jack Shafer, "The official excuses for the bad exit poll numbers don’t cut it." Slate Friday, Nov. 5, 2004, 9:23 PM PT One of the few people close to the pollsters to offer an explanation early on was Martin Plissner, former executive political director of CBS News (and self-described close friend of the pollsters), who identifies three problems with the polls: The pollsters who work outside the polling stations often have problems with officials who want to limit access to voters. Unless the interviews have sampled the entire day’s voters, the results can be demographically and hence politically skewed. Finally, it is of course a poll, not a set of actual recorded votes like those in the precinct samples collected after the polls close.18 Regarding the first problem, voters contacted in such precincts can be weighted. Jack Shafer of Slate observes: ... exit pollsters always encounter overzealous election officials enforcing electioneering laws. Can we really believe that a significant number of the 1,480 exit poll precincts in 50 states and the District of Columbia that Edison/Mitofsky surveyed on Election Day were so affected? And in sufficient numbers to bend state-by-state exit polls in Kerry’s favor?19 If in fact, pollsters have a limited sample of voters from a given precinct, they can weight the voters who are polled from that precinct or demographically similar precincts accordingly. Regarding time of day variation, this paper does not refer to mid-day reports, but rather end of day data, which happened to still be available at midnight. But even if there were an early voter bias, is there any reason to believe that early votes would be skewed Democrat? Stereotypically, Republicans are early risers. Regarding the last ditch argument that it’s just a poll, true (of course), but, as I have documented, the evidence and logic on exit polls suggest that we have every reason to believe they are accurate within statistical limits. Under-representation? Other explanations put forth by the Washington Post charge that samples may have included too many women, too few Westerners, not enough Republicans, etc ..." Regarding the first part of this critique, Morris writes: The very first thing a pollster does is weight or quota for gender. Once the female vote
reaches 52 percent of the sample, one either refuses additional female respondents or weights down the ones one subsequently counted. This is, dear Watson, elementary. 18 Martin Plissner "In Defense of Exit Polls: You just don’t know how to use them. Slate Thursday, Nov. 4, 2004 19 Jack Shafer, "The official excuses for the bad exit poll numbers don’t cut it." Slate Friday, Nov. 5, 2004, 9:23 PM PT Moreover, the issue of male/female ratio is irrelevant. CNN and others released data presenting male and female preferences separately, thus automatically weighting sex appropriately. Kerry voters apparently were much more willing to participate in the exit poll than were Bush voters. The interviewers at each of the sample voting locations are instructed to choose voters as they exit from the voting booth — every third, or fifth, or eighth voter — some sequence of this sort that is determined ahead of time. Many voters simply refuse to participate in the poll. If therefusers are disproportionately for one candidate or another, then the poll will be biased....21
OK, true enough. If Republicans disproportionately refuse to participate, that could explain exit poll error. But do we have any reason to suspect that? It is conceivable that Kerry voters were much more willing to participate in the exit poll than were Bush voters, but although it’s not difficult to imagine why some Bush voters might not want to participate, it’s also not difficult to imagine why some Kerry voters might not want to participate either. 20 Pollsters normally either choose precincts so as to ensure representative samples, or weight respondents by key demographic categories. The Utah Colleges Exit Poll website gives a fairly good basic explanation of polling practices: http://exitpoll.byu.edu/about/surve... and The problem with this "explanation" or even one that would have considerably more "face validity" (which means that it makes sense on the face of it), such as the possibility that absentee/early voters disproportionately favored Bush22, is that it is not an explanation, but rather a hypothesis. It’s apparent that "Kerry voters were much more willing to participate in the exit poll than Bush voters" only given several questionable assumptions. An explanation would require independent evidence.23 The Role of the Exit Poll Verifying the integrity of elections is important not only in Mexico, Venezuela and (the former Soviet Union Republic of) Georgia. Whatever the original purpose of this particular exit poll, it could be used to help verify the integrity of this election if its data and methodology were released. 23 I could imagine various types of supportive evidence. One possibility would be verifying sampled results versus actual voting patterns in random sample precincts where counts are unimpeachable. Summary and Implications My purpose in this paper, however, has not been to allege election theft, let alone explain it. Rather, I have tried to demonstrate that exit poll data is fundamentally sound, that the deviations between exit poll predictions and vote tallies in the three critical battleground states could not have occurred strictly by chance or random error, and that no explanations of the discrepancy have yet been provided. The unexplained discrepancy leaves us with two broad categories of hypotheses: the exit poll data are wrong (or misleading) in ways that hve not yet been adequately explained, or the count is off. The most important investigations concern verification of the tallies and allegations of fraud on one side; and examination of the exit poll data and methodology on the other. Some useful statistical analyses would compare the "shift" in (a) battleground states vs. non-battleground states; and (b) in states, counties and precincts where safeguards are strong vs. those where they are suspect. Obviously, if the polling consortium would release their data, that would allow us to do more definitive analyses. Given that neither the pollsters nor their media clients have provided solid explanations to the public, suspicion of fraud, or among the less accusatory, "mistabulation," is running rampant and unchecked. That so many people suspect misplay undermines not only the legitimacy of the President, but faith in the foundations of the democracy. Systematic fraud or mistabulation is a premature conclusion, but the election’s unexplained exit poll discrepancies make it an unavoidable hypothesis, one that is the responsibility of the media, academia, polling agencies, and the public to investigate. END ----------------------- v00l Dr. Freeman is on the faculty of the University of Pennsylvania; his areas of expertise include resilience, innovation, and research methods. He obtained his Ph.D. from the Massachusetts Institute of Technology. Contact him at stfreema@sas.upenn.edu. http://www.appliedresearch.us/sf/
![]()
|
![]() |
![]() ![]()
Tuesday 10 - 16:58
by Francis Kendrick
![]() ![]()
Monday 2 - 13:34
by Surprised
![]() ![]()
Tuesday 6 - 16:48
by Roger Moses
![]() ![]()
Wednesday 28 - 10:25
by Roberto Ferrario
![]() ![]()
Saturday 10 - 05:44
by David R. Hoffman, Legal Editor of Pravda.Ru
![]() ![]()
Wednesday 7 - 17:50
![]() ![]()
Friday 26 - 20:45
by BenAMarine
![]() ![]()
Thursday 25 - 16:32
![]() ![]()
Saturday 20 - 00:36
by David R. Hoffman
![]() ![]()
Thursday 18 - 01:55
by JACK RANDOM
![]() ![]()
Sunday 14 - 20:08
by JACK RANDOM
![]() ![]()
Friday 12 - 14:56
by Arty Kraft
![]() ![]()
Wednesday 10 - 11:10
by Anthon Lewis
![]() ![]()
Friday 29 - 17:44
by VincentT
![]() ![]()
Friday 22 - 00:06
by David R. Hoffman, Legal Editor of Pravda.Ru
![]() ![]()
Friday 15 - 12:03
by Kerim
![]() ![]()
Sunday 10 - 12:45
by Eaton
![]() ![]()
Wednesday 6 - 18:40
by JOHN CHUCKMAN
![]() ![]()
Saturday 18 - 01:38
by David R. Hoffman, Legal Editor of Pravda.Ru
![]() ![]()
Friday 10 - 15:13
by JOHN CHUCKMAN
![]() ![]()
Wednesday 8 - 09:22
by Bailey Anderson
![]() ![]()
Monday 30 - 19:43
by JOHN CHUCKMAN
![]() ![]()
Tuesday 17 - 23:07
by David R. Hoffman, Legal Editor of Pravda.Report
![]() ![]()
Tuesday 10 - 23:14
by David R. Hoffman, Legal Editor of Pravda.Report
![]() ![]()
Monday 2 - 09:55
![]() ![]()
Monday 2 - 09:47
![]() ![]()
Tuesday 12 - 14:22
![]() ![]()
Saturday 9 - 03:27
by lishk
![]() ![]()
Friday 8 - 21:44
by NathanT
![]() ![]()
Friday 8 - 21:37
by Nathan Taylor
![]() ![]()
Sunday 27 - 20:02
by David R. Hoffman, Legal Editor of Pravda.Report
![]() ![]()
Friday 25 - 20:38
by Perseus
![]() ![]()
Thursday 17 - 19:40
by celbbetty
![]() ![]()
Wednesday 16 - 01:44
by David R. Hoffman, Legal Editor of Pravda.Report
![]() ![]()
Thursday 10 - 12:36
![]() ![]()
Thursday 3 - 06:19
by Dwarika Nath Rath
![]() ![]()
Wednesday 2 - 03:10
by Dr Gideon Polya
![]() ![]()
Thursday 27 - 23:23
by David R. Hoffman
![]() ![]()
Tuesday 18 - 15:48
by Curtis Judge
![]() ![]()
Thursday 13 - 02:43
by Dr Gideon Polya
![]() |
Monday November 15 - 23:14 - Posted by 8f2e526d45560845...
Thank you for writing this. This work has lent credibility to what many have been feeling.
I think the majority of the American people see what is going on now. We cannot trust our government. We cannot trust our media.
But we have eachother, and the internet....and there are more of us than there are of them so HA! Democracy is alive and well. It is just a matter of time before their ship sinks, and the rats are already jumping.
Monday January 10 - 05:28 - Posted by a690c9cc034f5178...
I glanced at Freeman’s paper and noticed that he does not mention the fact that the exit polls were not only significantly off from the final tallies, but they were also off from all of the previous polls. He never mentions the previous polls at all, in fact. you cannot evaluate the exit polls alone like that.
I will go out on a limb and offer some speculation, but with better grounds than has been offered from the left: my speculation is that individuals who had a preference for one candidate attempted to skew the polling data early so that the voters for the other candidate would be discouraged from voting.
That the exit poll numbers were skewed, and badly, was obvious to anyone who had been visiting the various ’electoral vote prediction’ sites that were out on the web.
Honestly, the exit polls had Virginia for Kerry and South Carolina as too close to call. Was there even ONE electoral college predictor who made such a claim, ever? No.
The polling data never suggested it. Clearly the exit polls were skewed.
Who would benefit from elections being called one way early? There are two schools of though on that. One school is that it would motivate the supporters of the ’loser’ to get out in bigger numbers, while the other school thinks that it would demoralize the ’loser’s’ supporters and cause them not to vote. The second school got some support when Bush supporters in Florida reported left the polling places in western counties of the state after the state was called for Gore.
Thursday January 13 - 06:18 - Posted by 5163d92b0be10134...
"The second school got some support when Bush supporters in Florida reported left the polling places in western counties of the state after the state was called for Gore."
Hmmm- but it must have had the opposite effect in Ohio, where Warren County was in the western part of the state and one of the last counties to report. That’s where the Republican elections board said that received a homeland security threat, a ’10’ on a scale of 1-10- so they shut out independent observers (the FBI and Homeland Security both denied issuing such a warning)- and it just so happened that Bush scored an amazing 75% -25% blowout with the secret vote count, giving him +40,000 votes and sealing the win.
How convenient that the ’exit poll tricks’ by democrats chased republicans away in Florida, but lucky for you, it brought them out in record numbers in Ohio. Sure.