Community
The cost of dirty data – data that is inaccurate, incomplete or inconsistent – is enormous. Earlier this year, Gartner reported that, on average, poor quality data cost an organisation $15 million in 2017. These findings were reinforced by MIT Sloan Management Review, which reported that dirty data costs the average business an astonishing 15 to 25 per cent of revenue.
With global revenues of around $80bn per year, just in investment banking, this means that the cost of dirty data in financial services is astronomical. So, where does it come from and what can be done about it?
What’s the source?
Human error is a significant source. An Experian study found human error influences over 60 per cent of dirty data. When different departments are entering related data into separate data silos, without proper governance, fouling of downstream data warehouses, data marts and lakes will occur. Records will be duplicated with data, such as misspellings of names and addresses. Data silos with poor constraints will lead to dates, account numbers or personal information being shown in different formats, making them difficult or impossible to automatically reconcile.
Furthermore, once created, dirty data can remain hidden for years, which makes it even more difficult to detect and deal with when it is actually found. Most businesses only find out about dirty data when it’s reported by customers or prospects—a particularly poor way to track down and solve data issues.
And, still in 2018, dealing with print is an issue for many financial services firms. The scanning, marking up and import of printed documents is a recipe for the introduction of errors.
Many organisations search for inconsistent and inaccurate data using manual processes because their data is decentralised and in too many different systems. Harvard Business Review reports that analysts spend 50 per cent of their time searching for data, correcting errors and seeking out confirmatory sources for data they don’t trust. These plans tend to fall into the same trap as the data—instead of consolidated planning, each department is responsible for its own data inaccuracies. While this may work in some instances, it also contributes to internal inconsistencies between department silos. The fix happens in one place but not in another, which just leads to more data problems.
The impacts of Dirty Data
All of these issues result in enormous productivity losses and, perhaps worse, to a systemic loss of confidence in the data being used to power the business.
The estimates above of revenue loss because of poor data seem extraordinary, but even if they represent the upper limit of the true cost, the impact is still very significant.
In a highly regulated industry, such as financial services, dirty data has an even greater cost. Missing, incomplete and inaccurate data can lead to the wrong trade being made, decisions taking even longer as further manual checks are made and regulatory breaches being made. MiFID II has, of course, placed significant extra burdens on financial firms to ensure their data is in order.
Cleaning up the mess
What can be done? Here are a few things that organisations having difficulty with dirty data should be thinking about:
In conclusion, it’s worth stopping dirty data slowing you down. The business impact of dirty data is staggering, but an individual organisation can avoid the morass if it takes the right approach. Clean, reliable data makes the business more agile and responsive while cutting down on wasted efforts by data scientists and knowledge workers. And remember that 25% potential loss of revenue. It’s there to be clawed back.
This content is provided by an external author without editing by Finextra. It expresses the views and opinions of the author.
Oleksandr Strozhemin Co-founder and CEO at Trinetix
13 August
Elaine Mullan Head of Marketing and Business Development at Corlytics
12 August
Abhinav Paliwal CEO at PayNet Systems- A Neo Banking Software Platform
Donica Venter Marketing coordinator at Traderoot
Welcome to Finextra. We use cookies to help us to deliver our services. You may change your preferences at our Cookie Centre.
Please read our Privacy Policy.