LongRoom Unbiased - Methodology

The LongRoom Polling Analysis uses the latest voting data from each state's Secretary of State or Election Division. The voting data is kept current by incorporating the latest updates from each state as they become available. This means that the LongRoom Polling Analysis accurately reflects the actual voting demographics, precinct by precinct, county by county, and state by state.

Because the LongRoom Polling Analysis is exclusively data based, it makes it possible to demonstrate from the crosstabs of an individual poll whether that poll is either left or right leaning.

The analysis of the polls of each polling organization and the associated bias is illustrated in a line chart. The most recent poll results are displayed separately and a graphic representation of the amount the poll leans either left or right is shown.

The graphs below cover the last three presidential elections and show the LongRoom Polling Analysis of polls for those elections. In all cases, the LongRoom Analysis was accurate to within +/- 0.3%.

References for the voting data from each state are included below in the list of sources.

How do we know the polls are biased?

We know the polls are biased because the statisticians who produce the polls say they are biased, both explicitly and implicitly. This is also widely reported in the media. Let's look at two recent examples. The Reuters/Ipsos poll last week, July 29th, decided to use "forcing" to assign those who were surveyed to a candidate, even if the person who was surveyed had no preference. Reuters/Ipsos applied this "technique" not only to their most recent poll, but went back through all their previous polls and redid them, assigning those with no preference to a candidate of the pollster's choice. This innovative approach to polling was not universally popular with other pollsters, as Pat Caddell, a pollster with decades of experience, expounded in this article: "Pat Caddell on ‘Cooked’ Reuters Poll: ‘Never in My Life Have I Seen a News Organization Do Something So Dishonest’" . Another example would be the CNN poll from July 30th, where the crosstabs for Question P1 show that 97% of Democrats have committed to a candidate three months before the election. In the history of elections, it is difficult to find an example where 97% of a demographic have made up their minds on who to vote for even on election day, no less in the middle of summer before an election in November.

For a rather extensive list of biases that a statistician may introduce into a poll, there is an excellent article here by Nate Silver where he discusses the biases he uses in creating his analysis, and why he thinks his biases are good.

Statisticians also use "weighting" to produce the poll results that are published in the media. The weighting is simply how many of each demographic the statistician believes will vote based on the detailed questions that are asked when the poll is taken. An example of how this affects polls is demonstrated in the polls out this last week, ending July 31st. Some polls have changes of 10% and more in presidential preference while other polls have a change of only a few percent. Clearly, both of these results cannot be correct.

So like opinions, every statistician has their own biases, but none of them wants to see the other guy's. Here at LongRoom we leave out the biases and let the data speak for itself.

How do you remove the bias in the polls?

As we discussed above, each poll reflects the biases of the statisticians who prepare the poll. Since each statistician has their own specific biases that they introduce into their poll, it is extremely difficult to compare one poll to another. At LongRoom we use the actual state voter registration data from the Secretary of State or Election Division of each state. We add no "expert" adjustments to the data. This means that all the polls are rationalized one to another based on actual data.

What can we expect going forward?

As the election approaches, the statisticians who produce the various polls will begin to back out their biases. In the final few weeks before the election, you will start to notice a convergence of all of the polls. This occurs because the statisticians will use essentially the same data that LongRoom is using now to produce their polls with their own biases removed. So, you might be thinking at this point, are you really saying that all of the polls will eventually match LongRoom? Yes, we are, it is a mathematical certainty, that as the election approaches, all of the polls will begin to match the polls here on LongRoom. This may be difficult for some to believe, however, there is an excellent archive at RCP that shows the poll results for the 2012 presidential election and this typical convergence of polls as the statisticians' biases are backed out.

What is the earliest that you can tell who will be the next president?

The day after the election. This may sound humorous but it is actually the truth, there is no reliable predictor for who will win a democratic vote. An example of this is the March 14th, 2004 Spanish General Election which we covered and analyzed. On March 10th, 2004, the Conservative Party was leading in the polls, and as they were the incumbents, were likely to succeed in the election. However, on March 11th, there was a Madrid train bombing. The Conservative government quickly blamed the ETA Spanish Separatist group. As more information was uncovered, it became obvious that the bombing was the work of the Islamist group, Al-Queda. The Conservative government continued to claim it was ETA in spite of the mounting evidence. The electorate rapidly came to believe that the Conservative government was trying to cover up the Islamic involvement and gave the liberal opposition party a 5 point margin of victory. So, in a matter of only three days, there was an 8 point swing in voter preference.

For more information about the 2004 Spanish General Election and the impact the bombing had on it, Wikipedia has a write up here.

To make this example more relevant to our current presidential election, imagine that 3 days before the election there is a terrorist incident here in America, and Mr. Obama and Mrs. Clinton place the blame on right-wing Christian extremists, while Mr. Trump blames radical Islamic terrorists. As the hours tick by, it becomes obvious that the terrorist incident is the work of radical Islamic terrorists, however Mr. Obama and Mrs. Clinton continue to deny the Islamic involvement. Just as in Spain, it is game, set, match, and Mr. Trump is the next president of the United States.

So, if anyone pretends they can predict the election, just keep in mind: Life Happens.

What tools does LongRoom use?

We have developed our analytical model using the programming language that we and other actuaries have used for the last 30 years, APL.


References


  1. Alabama http://www.alabamavotes.gov http://www.alabamavotes.gov/ElectionInfo.aspx?a=voters
  2. Alaska http://www.elections.alaska.gov/statistics/vi_vrs_stats_party_2015.04.03.htm
  3. Arizona http://www.azsos.gov/elections/voter-registration-historical-election-data
  4. Arkansas http://www.sos.arkansas.gov/elections/Pages/default.aspx
  5. California http://www.sos.ca.gov/elections/
  6. Colorado http://www.sos.state.co.us/pubs/elections/VoterRegNumbers/VoterRegNumbers.html
  7. Connecticut http://www.sots.ct.gov/sots/lib/sots/electionservices/registration_and_enrollment_stats/nov15re.pdf
  8. Delaware http://elections.delaware.gov/services/candidate/regtotals.shtml
  9. Florida http://dos.myflorida.com/elections/data-statistics/voter-registration-statistics/voter-registration-monthly-reports/voter-registration-current-by-county/
  10. Georgia http://elections.sos.ga.gov/Elections/voterhistory.do
  11. Hawaii http://elections.hawaii.gov/resources/ http://elections.hawaii.gov/resources/registration-voter-turnout-statistics/
  12. Idaho http://www.sos.idaho.gov/elect/
  13. Illinois https://www.elections.il.gov/ElectionInformation/DownloadVoteTotals.aspx
  14. Indiana http://www.in.gov/sos/elections/ http://www.in.gov/sos/elections/2393.htm
  15. Iowa https://sos.iowa.gov/elections/voterreg/county.html#2016
  16. Kansas http://www.kssos.org/elections/elections_registration_voterreg.asp
  17. Kentucky http://elect.ky.gov/statistics/pages/registrationstatistics.aspx
  18. Louisiana http://www.sos.la.gov/ElectionsAndVoting/Pages/RegistrationStatisticsStatewide.aspx
  19. Maine http://www.maine.gov/sos/cec/elec/data/index.html
  20. Maryland http://www.elections.state.md.us/voter_registration/stats.html
  21. Massachusetts http://www.sec.state.ma.us/ele/eleenrollmentstats/enrollmentstats.htm
  22. Michigan http://www.michigan.gov/sos/1,1607,7-127-1633---,00.html
  23. Minnesota http://www.sos.state.mn.us/election-administration-campaigns/data-maps/ http://www.sos.state.mn.us/election-administration-campaigns/data-maps/registered-voter-list-requests/
  24. Mississippi http://www.sos.ms.gov/elections-voting/pages/voter-registration-information.aspx
  25. Missouri http://www.sos.mo.gov/elections/s_default/results
  1. Montana http://sos.mt.gov/elections/index.asp
  2. Nebraska http://www.sos.ne.gov/elec/
  3. Nevada https://www.nvsos.gov/index.aspx?page=1544
  4. New Hampshire http://sos.nh.gov/Elections.aspx
  5. New Jersey http://www.njelections.org/election-information-archive-2016.html
  6. New Mexico http://www.sos.state.nm.us/Elections_Data/2016-voter-registration-data-1.aspx http://www.sos.state.nm.us/uploads/FileLinks/c2426e22e02d478ca940282b232f0647/STATEWIDE_JAN_29_2016.PDF
  7. New York http://www.elections.ny.gov/EnrollmentCounty.html
  8. North Carolina http://www.ncsbe.gov/ http://enr.ncsbe.gov/
  9. North Dakota https://vip.sos.nd.gov/PortalList.aspx https://vip.sos.nd.gov/pdfs/portals/votereg.pdf
  10. Ohio http://www.sos.state.oh.us/sos/elections.aspx
  11. Oklahoma https://www.ok.gov/elections/Voter_Info/Voter_Registration_Statistics/
  12. Oregon http://sos.oregon.gov/elections/Pages/electionsstatistics.aspx#2016
  13. Pennsylvania http://www.portal.state.pa.us/portal/server.pt/community/voter_registration_statistics/12725
  14. Rhode Island http://sos.ri.gov/divisions/elections
  15. South Carolina http://www.scvotes.org/statistics
  16. South Dakota https://sdsos.gov/elections-voting/upcoming-elections/voter-registration-totals/voter-registration-comparison-table.aspx
  17. Tennessee http://sos.tn.gov/elections/election-information
  18. Texas http://www.sos.state.tx.us/
  19. Utah https://elections.utah.gov/election-resources
  20. Vermont https://www.sec.state.vt.us
  21. Virginia http://elections.virginia.gov/resultsreports/registration-statistics/2016-registration-statistics/index.html
  22. Washington https://www.sos.wa.gov/elections/vrdb/
  23. West Virginia http://www.sos.wv.gov/elections/history/Pages/Voter_Registration.aspx
  24. Wisconsin http://www.gab.wi.gov/elections-voting/statistics
  25. Wyoming http://soswy.state.wy.us/Elections/Docs/VRStats/2016/16JanVR_Stats.pdf