Event report - Scotland’s referendum: statistical perspectives

Written by Russell Ecob & Ben Torsney on . Posted in Features

Our event - 'Scotland’s Referendum: Statistical Perspectives', hosted by the Glasgow and Edinburgh local groups of RSS and BIS, was held at Edinburgh University on March 26. It was introduced by the Vice principal, Charlie Jeffrey, and comprised four speakers each tackling different areas (health, economic, production and dissemination of statistics, analysis of voting trends) relevant to the coming referendum on Scottish independence. A large audience of over 100 attended for the duration.

David Walsh of the Glasgow Centre for Population Health (GCPH), talked on the subject of ‘Scotland: health, its determinants, and the potential implications of independence’, in a ‘data-rich’ presentation. He started by providing an overview of health in Scotland, both in terms of comparisons with elsewhere in Europe, and in terms of the most pertinent issues within the country itself.

He asked the two questions. How fast is health in Scotland improving (in relation to other countries both in Western and Eastern Europe)? For whom is it improving? He also discussed what is known about the principal determinants of good and bad health in any society, and the potential relevance of independence to those factors in a specifically Scottish context.

How fast? He examined a range of health outcomes - infant deaths, working age mortality and life expectancy, ischaemic heart disease, lung cancer liver disease. Life expectancy (for both males and females) at birth tells the starkest story. In 2009, Scotland is amongst the lowest in Western European countries giving Scotland the unfortunate moniker of ‘the sick man of Europe’. He showed that this was not always so using data going back to mid-19th century. However since 1950 not only has life expectancy in Scotland fallen to the lowest of the Western European countries but also Scotland fared badly, in progress over time, since then in relation to Eastern European countries which had initially higher mortality rates.

For whom? He showed inequalities by deprivation category in Scotland, these increasing (in Greater Glasgow) over a recent twenty year period with male life expectancy in the highest deprivation quintile actually worsening over this period. He then went on to consider the reasons for this with reference to a well-used model (Dahlgren and Whitehead) of population health to which he added an extra domain ‘factors relating to the political economy’. Inequalities (in education and income and more generally) are now recognised as a driver of poor population health. He provided a range of evidence on the increase to these inequalities in Scotland over recent times.

Glasgow a special case? David pointed up recent research by GCPH which compares Glasgow with Liverpool and Manchester (cities chosen to have comparable industrial heritage) in which premature mortality is greater in Glasgow, particularly in the more deprived half of the population. Many theories have been put forward to explain this difference (the so-called Glasgow effect) and he pointed to some recent work (with Dr Gerry McCartney) on the possible so called ‘democratic deficit’, resulting from UK neoliberal policies from 1979 onwards, as a contributor.

He concluded that independence could influence Scotland’s poor health status but this would depend crucially on the policies adopted.

Jim and Margaret Cuthbert (independent statisticians and economists) talked on the economic aspects of independence. They put forward the view that, to a large extent, the debate on Scottish independence is conditioned by the choice of what statistics have been collected, the methods of analysis used, and the way the figures are presented.

They illustrated this with reference to a number of examples; the annual Government Expenditure and Revenues Scotland (GERS) exercise, the recent study produced by the Institute for Fiscal Studies, and the presentation of debt statistics in the light of quantitative easing and the Scottish Government's claim that the UK is in fact an optimum currency area.

GERS is an annual publication, first produced in 1992, under the eagis of the then Conservative UK government. The Cuthberts made a number of criticisms of this exercise summing up with the statement - ‘the existence of GERS, and the failure to produce (up until very recently) any coherent account of the revenues in and out of Scotland which are not related to government has had a profound effect in channelling political debate’.

They went on to consider the Office of Budget Responsibility (OBR) forecasts for the UK economy and the Institute of Fiscal Studies (IFS) Reports on the Scottish economy. They argued that OBR forecasts understate UK risks, and so the IFS approach would ‘immediately shift the debate to the risks surrounding the Scottish economy’ .

Finally they considered the report of the Fiscal Commission set up by the present Scottish Government which concluded that Scotland was close to being an ‘optimal currency area’ with the rest of UK. They made a plea for more refined analysis, acknowledging in particular the ‘marked inter-regional and inter-group disparities’ within the rest of UK.

Roger Halliday, chief Statistician of Statistics Scotland, talking on ‘The production of official statistics’, gave an overview of the integrated structure of analytical work in the Scottish Government and the context in which it sits, distinguishing between statistics produced in Scotland and statistics (at least at present) produced about Scotland from outside in terms of broad categories (health in former group, tax, economy amongst those in latter) .

He described the impact that statistical work has on a range of policy areas, and highlighted the key strategic developments statisticians producing official statistics in Scotland are taking forward. He maintained that, irrespective of the outcome of the referendum, ‘it is a fast moving time, with priorities being: doing more with data, modernising communication, strengthening our impact, developing partnership and maintaining trust in our statistics’. He gave his commitment ’to share any issues for official statistics that the independence referendum leads us to consider’.

He concluded that life is healthy in Statistics Scotland and that there are many opportunities for talented statisticians (irrespective of referendum outcome!).

John Curtice of the University of Strathclyde talked about ‘the performance of the polls’ by Skype from his home due to an acute back problem. He began by noting that, ‘unsurprisingly, many an opinion poll is being conducted during the referendum campaign, yet the challenge facing the pollsters is a considerable one’. Polls are having to estimate voters’ propensity to engage in a behaviour that they have not previously measured, to do so for a population where the practice of regular political polling stopped a decade ago, and at a time when the opportunities and challenges of internet polling remain the subject of methodological debate.

He gave a number of reasons why polls matter and thus their accuracy is potentially a subject of concern. Polls can affect the extent and nature of media coverage, the ability of campaigns to raise resources and activists and the tactics and arguments that the campaigns pursue.

At first glance the polls’ estimate of referendum vote intentions look highly volatile. However, the average level of support for 'Yes' and 'No' over time has, until recently, been remarkably stable, with just over 60% saying they would vote 'No' (after 'Don’t Knows' are excluded) and a little under 40% indicating they would vote 'Yes'. Since the New Year, though, the 'No' lead has narrowed somewhat.

The reason for this combination of volatility and stability is the existence of substantial ‘house effects’. Different polling companies have consistently reported very different estimates of the relative strength of the 'Yes' and 'No' camps. So two polls conducted at the same time have often produced different estimates of the level of 'Yes' and 'No' support. While at the same time, both have been suggesting that those levels were much the same as the last time the same polling firms reported.

The highest levels of support for 'Yes' have come from Panelbase, followed by ICM and Survation. Much lower estimates have been produced by TNS BMRB, YouGov and (especially) Ipsos MORI.

Curtice then described the differences between the polling organisations in the way in which interviews were obtained, the weighting and filtering schemes that they use and the question used to ascertain voting intention. Four of the six organisations polling regularly use non-probability internet samples derived from people who have previously agreed to join a panel of potential interviewees, one uses face to face quota sampling and one uses random digit dialling to obtain interviews over the phone. All but one of the companies weight their data by how people said they voted in the 2011 Scottish Parliament election.

All of these approaches can be questioned methodologically, but there is no straightforward correlation between method and whether a company typically reports a relatively high or a relatively low 'Yes' vote - though it should be noted the internet samples typically requited bigger weights to make them appear demographically representative. He concluded that ‘some polls (at least) are set to be wrong’ but would not be drawn on which ones.

Scottish referendum 2014 Glasgow local group Edinburgh local group

Join the RSS

Join the RSS

Become part of an organisation which works to advance statistics and support statisticians

Copyright 2019 Royal Statistical Society. All Rights Reserved.
12 Errol Street, London, EC1Y 8LX. UK registered charity in England and Wales. No.306096

Twitter Facebook YouTube RSS feed RSS feed RSS newsletter

We use cookies to understand how you use our site and to improve your experience. By continuing to use our site, you accept our use of cookies and Terms of Use.