Forthcoming in Foresight

In March 2004, Scott Armstrong, Alfred Cuzán, and Randy Jones launched PollyVote.com (Cuzán et al., 2005). The goal of this project was to use the high-profile application of election forecasting to demonstrate advances in forecasting methodology. The primary focus was the “principle of combining”, in which one can achieve large reductions in forecast error by combining forecasts from different methods that draw upon different data. Armstrong (2001) has summarized evidence from 30 empirical comparisons. He found that combining nearly always reduced the error from the typical individual method. The average error reduction was 12.5 percent.

Computation of the PollyVote

During the past two U.S. presidential elections, in 2004 and 2008, Polly the Parrot, the mascot of the PollyVote, provided predictions of the two-party popular vote-shares by averaging forecasts within and across four component methods: trial-heat polls, forecasts from the Iowa Electronic Market (IEM), quantitative models and experts’ forecasts.

Combining within component methods was used to aggregate forecasts based on a similar approach and similar information. It is, however, more powerful to average across methods, especially when the methods draw upon different information. Using equal weights, the final PollyVote forecast is computed by averaging forecasts across the four component forecasts.

Past performance

With this simple averaging procedure, the PollyVote has provided highly accurate forecasts of election outcomes. In 2004 Polly posted updated forecasts every two or three days.  An automated system developed in 2008 enabled Polly to provide daily forecasts.  In both elections PollyVote forecasts were made beginning more than half a year before Election Day. Each single PollyVote forecast -- even those generated several months before the election -- has been correct in predicting the election winner.  In the final forecasts released on Election Eve, Polly missed the candidates’ actual two-party vote-shares by 0.3 percent in 2004 and 0.7 percent in 2008 – an average error of only 0.5 percentage points.

Graefe et al. (2011) summarize the predictive performance of the PollyVote for the past five U.S. presidential elections, reporting retrospective analyses of the 1992, 1996, and 2000 elections, in addition to the ex ante forecasts for 2004 and 2008. They found that the PollyVote forecasts reduced error by 52% to 58% compared to forecasts of the typical randomly chosen poll, model, or expert. Compared to the IEM, which is essentially a means of aggregating information from disparate sources, the PollyVote reduced error by 10%.

PollyVote 2012

To forecast the 2012 presidential election, Polly added a new component, models that are based on the index method. This method, which has a long history in forecasting and decision-making, is useful for identifying the best of several options. It draws upon prior evidence to determine which variables are important and how they affect the outcome. The method works well when there are many important variables and good prior knowledge (Armstrong & Graefe, 2011).

The first index model used for presidential election forecasting was the “Keys to the White House” by Allan Lichtman (2010), which first appeared before the 1984 election. As noted by Nadeau and Lewis-Beck (2012) in this issue, the Keys forecast the election winner by assessing how well the party in the White House has governed the country. Since the Lichtman index first appeared, other index models have been developed that predict the election outcome based on information about candidates’ biographies or voters’ perceptions of candidates' ability to handle issues facing the country. Detailed information on each of the models can be found at PollyVote.com.           

Indexes are a proven forecasting method and draw upon different information than traditional approaches to election forecasting. Therefore, including them as a separate component of the PollyVote is expected to further increase Polly's forecast accuracy.

Year-ahead forecast for 2012

On January 1, 2011, almost two years before Election Day, the PollyVote was launched to forecast the 2012 presidential election. Since then, the daily updated forecast has consistently predicted Obama to win the two-party popular vote, with a vote-share forecast ranging from about 54% shortly after the death of Osama bin Laden in May, to almost 50% in early November.

As of mid-November 2011, the combined PollyVote forecast predicts Obama to win 51% of the vote. Thus the forecasts of the four component methods currently available suggest a close race. While the combined poll forecast shows a slight advantage for Obama (50.5%), the IEM (49.6%) and the average of five econometric models (49.5%) put him slightly behind the Republican candidate. Only the average forecast of four index models predicts a clear victory for Obama (54.4%).

These forecasts will, of course, continuously change during the run-up to the election. In particular, the selection of the Republican candidate may have a big impact. In addition, econometric and index models may not have been released yet, and results from other indicators will be updated as new data becomes available. Current polls and updated IEM prices are published almost daily – and generally become more accurate closer to Election Day. Finally, Polly is waiting for her fifth component method, the experts’ forecasts, which will be added as soon as available.

Summary

The PollyVote combines forecasts within and across virtually all methods that are commonly used in election forecasting. This is beneficial because large gains in accuracy can be achieved by combining forecasts from different methods that draw upon different data. Instead of choosing a single forecast, Polly relies on a combination of several forecasts in order to avoid large errors. Polly thus demonstrates through election forecasting the validity of the "principle of combining" for forecasting problems more generally.

References

Armstrong, J. S. (2001). Combining forecasts. In: J.S. Armstrong (Ed.), Principles of Forecasting: A Handbook for Researchers and Practitioners. Boston: Kluwer Academic Publishers, pp. 417-439.

Armstrong, J. S. & Graefe, A. (2011). Predicting elections from biographical information about candidates, Journal of Business Research, 64, 699-706.

Cuzán, A. G., Armstrong, J. S., and Jones, R. J., (2005).  How we computed the Pollyvote. Foresight, 1, 51-52.

Graefe, A., Armstrong, J. S., Jones, R. J. and Cuzán, A. G., Combining forecasts: An application to election forecasts. APSA 2011 Annual Meeting Paper. Available at SSRN: http://ssrn.com/abstract=1902850.

Lichtman, A. J. (2010). The keys to the White House: Forecast for 2012, Foresight, 18, 33-37.

Nadeau, R. and Lewis-Beck, M. S. (2012). Does a presidential candidate’s campaign affect the election outcome? Foresight, 24, XX-XX.