Data Governance is now a new normal but with Big Data?

There are enterprise horns honking the words that Big Data has the potential to drive Banking to a novel era. Being a millennial and an avid believer of business transformation driven by technology, I don’t think twice on a positive nod for this aspect.

However, Enterprise Data Governance has become a new normal for most large banks. Various aspects of data governance including classification, lineage, and data quality management have gained recent importance due to integration of Big Data into the enterprise data warehouses or data lakes. Business and IT, at the same time, are adopting evolving governance framework and policies.

In 2013, I was providing consulting services to a UK bank on setting up its data governance program. There were several lessons learnt in process discoveries to drive self-service, automation of quality management, identifying ownership, lean processes to cut down cycle time to assessment with the traditional enterprise in-house information. This when clubbed with the evolving needs hints us that Data governance is still an evolving model.

Integrating and profiling the newly acquired voluminous data is the first step to data quality assessment. The initial profiling results along with the decisive aspects of this data would help us establish the contextual perspective of Big Data to trust or Truth. Though, early profiling services were set up with a view of scalability to accommodate future growth of data, questions arises if the assessment and continuous data quality measurement environments are supportive of the voluminous increase in data being integrated from different external sources and the ever increasing process metadata. There are solutions readily available in the market including Informatica Big Data edition that can easily integrate, profile, transform data on Hadoop or between source and target systems.

Though banks have caught up with Data governance early this decade, their main focus is to understand the business levers and value found in achieving data quality that in turn drives Value and return. While Business analysts are busy identifying Business cases, IT teams are geared up for infrastructural and architectural changes following the enterprise adoption of Big Data. The data centers of the sales & Marketing, Operations, Risk and Compliance divisions are increasing multi-fold as most of the investments, Discoveries and Value are driven in these areas. The data generated from extended channels, third parties and publicly available sources is burgeoning and is being integrated into internal structures. This presents several opportunities and challenges to the CDO for expanding the Data governance umbrella to these Big Data Sources as well.