A new poll by Savanta ComRes for Scotland on Sunday has found a 4% lead for the union. Only it hasn’t – in the newspaper itself the article carries this incredible admission:
‘’However, these figures are not weighted for voter turnout, with further polling expected this week to show a clearer impact of the inquiry on Scottish independence voting intention.’’
These figures on Scottish independence are not directly comparable with previous polls on the subject, due to this and the nature of the poll.
Weighting the result by intention to vote is industry standard practice and is easily done. It can make a significant difference to the results. So not only can this Yes/No split not be compared to other previous polls it can’t even be taken seriously.
There is also a properly weighted Panelbase poll out in the Sunday Times that has Yes: 50% (-3) No: 50% (+3) and so there is a clear trend caused by the Salmond Enquiry that support for independence has dropped – the question is by how much? Scotland on Sunday manipulated the figures to have a better story and it is effectively lying.
I am not saying the poll would be better for one side or the other if the weighted results were presented – it might be even worse for the Yes side. It is even technically feasible that it would make no difference but as it stands the numbers are of no use in terms of drawing any meaningful conclusions.
I have commissioned multiple polls in the last 18 months and I don’t even look at the unweighted results as they are effectively meaningless. So why publish a deferent methodology to all other polls unless you are trying the skew the results?
I called the Scotsman head office and also Savata ComRes and even some mobile numbers I was able to finder for the journalist himself and no one answered. If I published a polling story on a Sunday that was such big news I would make someone available for comment and both those organisations employ a lot more people to answer questions.
Britain elects a totally non partisan polling tracking organisation has now deleted Tweets reporting the Scotland on Sunday poll and calling the methodology curious.
So we used our Twitter account @BisforScotland to ask the key questions that really need answered about today’s poll in Scotland on Sunday directly to @conor_matchett the author of the article and to @SavantaComRes the polling company.
Here are the 12 Tweets we sent asking for answers.
1: The Scotland on Sunday poll on independence has a methodology note that makes no sense – we need answers @conor_matchett @SavantaComRes #ScotonSunday #ScottishIndependence #poll
2: @conor_matchett @SavantaComRes Can you say why Scotland on Sunday ran a poll story with a non-standard methodology? #ScotonSunday #ScottishIndependence #poll
3: @conor_matchett @SavantaComRes Did you get the weighted numbers first and decide to publish the non-weighted numbers because it made a better story? #ScotonSunday #ScottishIndependence #poll
4: @conor_matchett @SavantaComRes Can you confirm the numbers were weighted against the recalled vote in the 2019 General election. If so why not weight against the intention to vote?#ScotonSunday #ScottishIndependence #poll
5: @conor_matchett @SavantaComRes What difference did it make compared to the weighted figures? #ScotonSunday #ScottishIndependence #poll
6: @conor_matchett @SavantaComResWhen will you publish the tables so others can do the work and provide the weighted numbers? #ScotonSunday #ScottishIndependence #poll
8: @conor_matchett @SavantaComRes You have suggested that the weighting wasn’t done as you are doing further investigations. This makes no sense. Can you explain? #ScotonSunday #ScottishIndependence #poll
9: @conor_matchett @SavantaComRes Did you forget to ask the question on voting intentions or do you not know how to do the weighting?#ScotonSunday #ScottishIndependence #poll
10: @conor_matchett @SavantaComRes If the weighting showed a worse picture for the Yes number why didn’t you use that? #ScotonSunday #ScottishIndependence #poll
11: @conor_matchett @SavantaComRes Why is a poll using methodology that renders it inaccurate a news story? Did the properly weighted numbers not tell as good a story for the papers editorial line? #ScotonSunday #ScottishIndependence #poll
12: @conor_matchett @SavantaComReWill the Scotsman have a follow up story publishing the Holyrood voting intentions from the poll tomorrow – will those be weighted properly? If so, why wasn’t today’s poll? #ScotonSunday#ScottishIndependence #poll
We will update this article as and when we get any answers. As we have pointed out we don’t have access to the raw data so can’t actually say that this was a 50/50 poll or even had a Yes lead – technically the weighted figures could be even better for the union.
However it takes a mater of minutes to weight the intention to vote for the polling company and if those figures were worse for Yes then why would an ultra-unionist newspaper not use those less contestable figures.
It leaves the door open for accusations of fake news and falsifying numbers and manipulation of the news. The raw data tables must now be published under British Polling Council rules so we will soon know soon one way or another.