Big Data: Big Deal?
In part 1 of our series on big data, Intelex team member Tom Morrison-Bell looks at what we mean by Digital Government, Open Data and Big Data.
We are swamped with information. Eric Schmidt, executive chairman and former CEO of Google claims that we now create as much information in two days as we did from the dawn of civilization until 2003. This overabundance of information, or ‘information overload’, can be overbearing, disorienting and even paralysing.
While much of this information is trivial in itself, it can be collected into vast datasets – big data – which, when analysed, provide invaluable intelligence. Most of us are probably aware of this in the business world – somewhere in the back of our minds, at least, we sense that Google and facebook are crunching the petabytes of data they have to help companies sell us their wares – but what are the implications for Government and the public sector? That is the focus of this series.
This first piece briefly sets out the difference between the closely related terms ‘digital government’, ‘open data’ and ‘big data’. These can often be confused because of their close relationship, so a clear understanding of each term is important before examining how big data can help transform Government and public services.
Digital Government refers to the process of using digital, usually online, tools in the provision of Government services. This can be as mundane as replacing paper with digital alternatives. Consider driving licence applications. The DVLA receives two articulated truckloads of paper each day, which incurs a huge cost. Another excellent example is the Passport Office’s online application. The Office kindly prints out your online form, only to return it by post for you to sign. Policy Exchange, the leading think tank on all questions digital (including ‘big data’) suggests that moving almost all Government services online could result in savings of up to £70bn.
The Cabinet Office does currently have a ‘Government Digital Service’, set up in the wake of Martha Lane Fox’s 2010 report. The first services to go online include tax assessments and visa, apprenticeship and pension applications. Do keep an ear out for the term ‘digital by default’. This is the Government’s buzzword for digital reform.
Open data is the idea that certain data should be freely available for anyone to access, use and republish. The current Government is committed to the idea of open data and launched data.gov.uk which releases public data ‘to help people understand how Government works and how policies are made’. From a standing start in 2010, the UK is now one of, if not the, leading country in publishing government data online.
While this is praiseworthy, and shows that the Government is serious about open data, it’s not just about the datasets – it’s also about providing them in a usable form. The recent Institute of Government’s ‘Whitehall Monitor’ made exactly this point. The report stated of the data the Government publishes, ‘while we can get compelling insights on parts of the picture, it is currently impossible to comprehensively assess Whitehall’s effectiveness.’ Full marks to the Government for effort but it may still take a while to perfect.
Big data refers to datasets that are so large (and complex) that they are practically impossible to analyse using traditional, hands-on database management tools. Big data is closely related to open data – many, if not all, of the datasets on data.gov.uk would qualify as big data. The Government has access to gigantic datasets – patient records, DNA databases, HMRC data, welfare, and so on, which, according to Policy Exchange, could result in £16bn to £33bn of savings if effectively analysed.
There are a number of areas that big data can make a difference in Government but in order not to add (too greatly) to the information overload, the following pieces in this series will look at how big data could help transform the provision of services in health, welfare and also save large sums of public money through efficiency savings.
Part two of the series will focus on Big data: big savings? How much can the Government save through efficiencies?