For the Pollsters, it’s 1992 all over again

From the Political Betting blog - Ex-ICM boss & political polling pioneer, Nick Sparrow, on the 2015 polling "debacle"

May 8th, 2015

The pollsters agreed, it was too close to call, a hung parliament a near certainty. There were even signs in some polls of a swing to Labour over the last few hours and days. Ed looked increasingly confident, Dave a bit dejected. The pollsters polled as late as possible to catch late swing. YouGov re-contacted their final poll sample and found no reason to change their prediction. Overall the pollsters predicted the Conservatives would get 34%, Labour the same, the LibDems 9% and other parties 26%.

With the votes now counted, we can now see that they under-estimated the Conservatives by 4%, overestimated Labour by 3%, overstated the LibDems by 1% and understated others by 1%. The error in the estimation of the lead was 7%, the average error in the four estimates (Con, Lab, LibDem and Others as a group) of 2.25%.

It is all eerily similar to 1992. In what has previously been labelled the great polling debacle, the pollsters, on average underestimated the Conservative share by 4% and overestimated Labour by 4%, an error in the lead estimate of 8% and an average error only marginally higher than 2015 at 2.75%.

Of course, what really upset political commentators in 1992 was that the polls pointed to a hung parliament with a possibility that Labour might be the largest party; the wrong story altogether. They escaped similar criticism in 1997, not by being much more accurate, but because the headlines they produced were correct – a handsome Labour victory. In 2015, not only were the polls almost as inaccurate as in 1992 – again they pointed to a different outcome.

The inaccuracy of the polls in 1992 sparked a two year investigation by the Market Research Society and calls for polls to be banned, at least in the period immediately prior to an election. We can expect those demands to be repeated now. It will be said that polls might influence the choices people make, and frame political debate, and if they are plain wrong, their publication – especially at election time – cannot be justified.

As in 1992 there is talk of an investigation into the polls with the aim of finding out what went wrong and fixing the problems.

Trouble is, somewhere near to top of the list for that committee must come some questioning of sampling methods. Unquestionably, over the last 23 years it has become more difficult to achieve a sample that is really representative of all voters, and the desire for cheap polls has fuelled the dash to online methods. Assuring sceptics that telephone polls that get usable data from less than 1 in 5 of the numbers dialled, or satisfying those who question the reliability of online panels of voters who actively seek an opportunity to record their vote intentions will be a substantial challenge.

Nick Sparrow