It provides concrete examples of applying big data in developing economies and proposes a model for how all producers of official statistics can respond to big data's challenges and opportunities. One of the authors, Emmanuel Letouzé was passing through London on 26 February and spoke to a group of government statisticians to give his perspective. You can watch his presentation below and also see the slides here.
Emmanuel showcased some exciting current applications of big data. He showed how mobile phone data has been used in estimating population changes and movements, and the use of machine learning to model predictor variables to predict crime hot spots and estimate poverty. He highlighted a number of 'perspectives and requirements' - including technical challenges such as estimating the sample bias. This highlights a clear need to have a good understanding of where the data comes from.
For example, if using mobile phone data then a good understanding of mobile penetration, who is using the phones and how they use them is needed. He also talked about ethical and commercial challenges this presents before outlining how we need to think about official statistics as more than just the deliverables of our analysis, but also comprising the process and the people which deliver it.
In the white paper he descibes how official statistics not only has a dual nature, but it also serves two main functions: 1) Its first and probably most fundamental function is to produce knowledge. In the words of Enrico Giovanni (president of Italian national statistics), the essential role of modern statistics (referred to as 'statistics 2.0') is to provide society with:
'Knowledge of itself, on which to base its own choices and evaluate the effects of political decisions.'
The second function of official statistics is to provide a deliberative space where questions around what is worth measuring, how it is measured, and for which purpose it is measured are freely and openly debated - to act as 'a debated public institution'.
The role of statisticians therefore is not only to get to grips with new data sources and seek to harness them to be more effective and efficient, but also to help frame the debates as to how society as a whole can take advantage of these new sources of knowledge.
The views expressed in the Opinion section of StatsLife are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of The Royal Statistical Society.