Recent polling in the United Kingdom has proven to be wide of the mark, so how did they get it wrong with the Brexit results?

The historic vote saw the Leave campaign win out with a small majority of 51.9%. Although those statistics don’t sound particularly huge, there were 17.4 million votes cast for the Leave side, while the Remain side mustered 16.1 million.

Despite that difference of over one million people, Yougov had the Remain campaign down to win as the polls were closing on the Thursday night, while Ipsos Mori also had them ahead as the closing stages of the race approached. While they admitted they were a few points off, Yougov emphasised that it should come as no surprise that the referendum was a close run thing.  

As they noted in a post after the results became clear, they said "we do not hide from the fact that YouGov’s final poll miscalculated the result by four points. This seems in a large part due to turnout – something that we have said all along would be crucial to the outcome of such a finely balanced race."

The sheer number of people who did or didn't make it to their polling station does seem to have played rather decisive role, with particular regional divides becoming clear in the breakdown of the results. Scotland and Northern Ireland (as well as Gibraltar) overwhelmingly voted to stay, while a higher than expected turnout in the north of England saw the region outvoting the south. 

A HM Customs Gibraltar sign on the Gibraltar side of the Spanish border | Image: Ben Birchall / PA Archive/Press Association Images

Speaking to Newstalk.com, Richard Colwell of Red C Research stated that while they were out by more than the three point margin of error, the “shy voter” and “status quo” effects, which are normally accounted for in the polling, had a larger role to play than usual.

“The key thing is that the 'shy voter' effect was at play far great than the 'status quo' effect," said Colwell. "The pollsters can be forgiven for taking it into account, since it has been so apparent in previous referenda, so you could understand why they would think it would happen in the Brexit.

“However, It was cancelled out by a couple of things. Because of the issue around the poster from Nigel Farage and then the death of Jo Cox, the 'Leave' people were less inclined to tell pollsters that they were voting that way."

Colwell also noted that turnout, although higher than expected, was affected in certain areas: “Across the UK, the two areas that have been instrumental in the result, Scotland and London, saw a lower turnout, both of which were strong remain areas. Scotland was potentially to do with a lacklustre campaign, but in London you could talk about the weather. There were simply appalling conditions, and people couldn’t even get home to vote, which impacted the result."

With the pollsters being forced to make a prediction, coupled with a reticence from people to admit that they were going to vote ‘Leave’, it was then little surprise that the 'Remain' side was winning when predictions were made, not only by the pollsters, but also by the bookies and financial institutions.

That was not, however, the wisdom of the crowd, either figuratively or literally. Populus, who conducted a number of different polls in the run up to the referendum, came much closer to predicting the result, calling it for the 'Leave' campaign by 51% to 49%, using the 'Wisdom of Crowd' technique.

This involved asking people not only how they were going to vote, but also what way they thought the result would go. As Colwell notes, it proved to be extremely reliable in the marriage equality referendum here in Ireland, calling it for the 'Yes' side by the correct percentage.

This technique tends to show "people who either aren’t captured by the polls, or are trying to hide which way they will really vote," said Colwell. "They’ll say things such as ‘well my friends are going to vote this way’ and you get effectively a more unbiased picture of what's going on."

While online polls may similarly cut through that reluctance from voters, there is a caveat to relying too heavily on one technique over another, or failing to give one adequate weighting, depending on the issues at stake.

"'Wisdom of Crowd' polling is an extremely good approach for referenda, but we’ve tried it in general elections where it's less effective," said Colwell. With people less focused on one particular issue and considering more diverse factors, "they don’t sit round the kitchen table and talk about it, and they’re not as engaged."

With a reliance on polls and statistical punditry in the United States election, popularised by sites such as FiveThirtyEight, the questions over the increased effect of the 'shy voter' may yet come into play once again in the lead up to November's presidential election there.

However, with a parliamentary petition already calling for another referendum, and the possibility of the EU coming back to the United Kingdom with improved terms for their membership, there may yet be a chance to do this all again, giving the pollsters another chance to refine their techniques.