Data handling – A rethink

The European Organization for Nuclear Research’s (CERN’s) Large Hadron Collider produces truly enormous quantities of data. Each particle collision generates about one megabyte of data – not so large individually, but when the system is fully operational, 600 million events take place each second. This is too much data to be recorded in practice, even by CERN’s state-of-the-art computing network. But since, on average, only one in a million events is of interest, it’s possible to filter a lot out. Using algorithms that assess the likelihood of an event being of interest, CERN’s processors select 100,000 data points from the 600 million and pass these on to a second set of more specialised algorithms, which refine them to just a few hundred, which can be stored for analysis.

No central bank has to cope with anything close to this sort of data flow, but it highlights just how big data can force institutions to rethink the way they collect, process and manage data. 

This year’s survey of central banks finds that 61% of respondents view big data as a core or auxiliary input to policy – a small increase on last year. Many central banks say they are upgrading their internal data governance to better manage larger quantities of less standardised data, and a handful of governance upgrades was completed in the past year. 

There is still some way to go. Despite big data’s growing importance, 62% of respondents to the survey said that data governance is not clearly defined within their organisations. Our feature on the International Monetary Fund’s efforts to upgrade its own data governance and take advantage of big data may provide some useful ideas in this area. The fund is also working with many of its members to upgrade their data capabilities.

We also asked several regulatory technology specialists how big data can change the way financial systems are overseen, reshaping and hopefully streamlining the work of supervisors and private sector firms. Our forum participants from the Bank of England and Sveriges Riksbank expressed the hope that big data techniques could soon be embedded in the day-to-day operations of central banks.

Official institutions are maturing as big data users but, as is made clear by this year’s report, there is plenty more work to be done.

This feature is part of the Central Banking focus report, Big data in central banks,  published in association with BearingPoint.

Download the 2018 focus report in PDF format.