Top 10 challenges in building data warehouse for large banks

Lack of strategic focus to build Enterprise Data Warehouse (EDW)

Building EDW is a strategic initiative since it requires shift in culture, longer timescale & more importantly it is an expensive affaire. Hence, it should be one of the top agendas of the CXOs and they need to closely monitor the progress and also need to provide executive support to break any unwanted barriers.

Need of considerable Time, Effort & Cost

Typical time taken for a global bank to build an EDW varies from a couple of years to 5 years. It also requires substantial effort & eventually huge amount of money to build a data warehouse. Also, Evidence of successful ROI is very opaque in the existing data warehouse implementation.

Lack of cross divisional collaboration

Building EDW requires constructive collaboration from various teams like multiple business divisions, source system teams, architecture & design teams, project teams and vendor teams. 

Technological complexity

Mostly, source data is kept in multiple operating systems & multiple data base technologies. There are plenty of tools for data sourcing, data quality management, data integration, data ware housing, reporting & analytics. Choosing appropriate technology is not so simple and is complicated by various emerging techniques like data virtualisation,  self service BI, in-data base analytics, columnar data base, NOSQL database, massively parallel processing,  in-memory computing and etc,. Also, traditional data warehouse is required to be integrated with big data technologies & Internet of Things for gaining business insights.

Ill-defined, changing business data requirements & Insensitivity of technical team in understanding business requirements

Most of the time business finds difficulty in defining the data requirements, since data requirements keep evolving as the use of data increases. However, technical team wants finalised data requirements from business before designing & building a data warehouse.

Lack of clarity on true source of data

Most of the large banks have great legacy behind them and have been growing over decades through mergers & acquisition. They have widespread footprint across geographies and various customer segments. In this process, they have acquired many systems which are poorly integrated, less documented and data is scattered across multiple systems. It is nightmare for these banks to identify the true source of their data.

Lack of ability to manage data quality issues

Since data is an organisational asset it needs to be acquired & maintained well.

Many front office/customer facing systems don’t capture quality data at its origination. There is no unified data capturing process across organisation.

For example, last name of a personal customer would not have been captured in a front office system, since it is not a mandatory field, whereas it may be mandatory field for another system.

Sometimes there is lack of well defined processes & technologies to curtail the data quality issues.

Vested interest of vendors in promoting their own solution

Most of the top data warehousing vendors have their own suit of solutions/products in the entire data warehousing eco system. These vendors tend to promote their own solution rather than advocating what is best suited for the customer.

Comfort of using divisional data marts

Reporting is indispensable activity of banking. Many banks have built divisional data marts for fulfilling their own divisional needs. Though divisional marts do not provide enterprise wide view, many business users are comfortable in using divisional data mart assuming that “Known devil is better than unknown angel”.

Subordinate use of data ware house

Business users from various divisions need to use data warehouse for reporting, business intelligence, data analytics & advanced analytics to unleash full potential of the enterprise data asset. Under utilised data warehouse will not grow & will not yield the desired return on investment (ROI)