Big data and official statistics - the international view

Written by Emily Poskett on . Posted in Features

During January’s Royal Statistical Society's debate on 'What is the future of official statistics in the big data era?', John Pullinger described big data as 'a wakeup call for official statisticians'. Despite the huge challenges, there are indeed many who agree. Official statisticians around the world have been thinking about what big data means for them, and innovating in a wide range of fields, in all corners of the globe. An international conference on 'Big Data for Official Statistics' organised by the United Nations Statistics Division and the National Bureau of Statistics of China in late 2014 brought some of the movers and shakers together.


One of the most influential groups in this arena is the DataPop Alliance think-tank, which focusses on big data and development. They have produced a white paper entitled 'Official Statistics, Big Data and Human Development: Towards a New Conceptual and Operational Approach' (written together with Paris 21) which is currently available for comment.


It provides concrete examples of applying big data in developing economies and proposes a model for how all producers of official statistics can respond to big data's challenges and opportunities. One of the authors, Emmanuel Letouzé was passing through London on 26 February and spoke to a group of government statisticians to give his perspective. You can watch his presentation below and also see the slides here.

Emmanuel showcased some exciting current applications of big data. He showed how mobile phone data has been used in estimating population changes and movements, and the use of machine learning to model predictor variables to predict crime hot spots and estimate poverty. He highlighted a number of 'perspectives and requirements' - including technical challenges such as estimating the sample bias. This highlights a clear need to have a good understanding of where the data comes from.

For example, if using mobile phone data then a good understanding of mobile penetration, who is using the phones and how they use them is needed. He also talked about ethical and commercial challenges this presents before outlining how we need to think about official statistics as more than just the deliverables of our analysis, but also comprising the process and the people which deliver it.

In the white paper he descibes how official statistics not only has a dual nature, but it also serves two main functions: 1) Its first and probably most fundamental function is to produce knowledge. In the words of Enrico Giovanni (president of Italian national statistics), the essential role of modern statistics (referred to as 'statistics 2.0') is to provide society with:

'Knowledge of itself, on which to base its own choices and evaluate the effects of political decisions.'

The second function of official statistics is to provide a deliberative space where questions around what is worth measuring, how it is measured, and for which purpose it is measured are freely and openly debated - to act as 'a debated public institution'.

The role of statisticians therefore is not only to get to grips with new data sources and seek to harness them to be more effective and efficient, but also to help frame the debates as to how society as a whole can take advantage of these new sources of knowledge.


This article first appeared on the Government Statistical Service Data Blog.

The views expressed in the Opinion section of StatsLife are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of The Royal Statistical Society.

Big Data Official Statistics

Join the RSS

Join the RSS

Become part of an organisation which works to advance statistics and support statisticians

Copyright 2019 Royal Statistical Society. All Rights Reserved.
12 Errol Street, London, EC1Y 8LX. UK registered charity in England and Wales. No.306096

Twitter Facebook YouTube RSS feed RSS feed RSS newsletter

We use cookies to understand how you use our site and to improve your experience. By continuing to use our site, you accept our use of cookies and Terms of Use.