If you want data to bring real value, you should consider how it is stored and processed. Being obliged to follow the ever-changing regulations of national banks and ministries of finance, banks require proper data management. Central repositories of banks'
data, called data warehouses, must ensure this appropriate data management. However, most banks still host their DWHs on-premises and use legacy technology, which makes it costly to handle data and scale it as needed.
Here are some helpful recommendations to unlock the power of data, rethink data management strategy, and implement DWH successfully.
Tip 1 - Ask why you need it
Setting up project goals will facilitate a better understanding of why you need to implement a data warehouse. Discuss the outcomes you are expecting to get with all project stakeholders. At this stage, you should also consider the following things:
Usually, banks hire external DWH development providers or create hybrid teams consisting of in-house IT staff and external vendors to manage and execute the project. No matter what setup you have, discuss all the requirements, nuances, and risks with all
the people involved in the project.
Tip 2 - Analyze existing systems
Inventory of your data is a must in any data warehousing project. A data-driven culture usually requires proper data management and governance rules and standards. That is why it is instrumental to identify data sources, global dictionaries, and relational
database schemas. Besides, if you have multiple systems, ensuring their interoperability by capturing each system's semantics and creating a standardized framework for exchanging semantic specifications and querying is essential.
Tip 3 – Choose architecture
Your choice of architecture depends on your specific goals. You have three options: single-tier, two-tier, and three-tier architectures, with the three-tier option being the most utilized. Single-tier architecture is the best fit if your primary aim is to
remove redundant data and reduce stored information. If you require a separation of the data warehouse and available data assets, then the two-tier architecture is the right choice. However, the three-tier architecture for data warehousing stands out as the
most popular option due to its multi-modular structure, offering superior flexibility and data management capabilities. Insufficiently detailed architecture planning or excessive redundancy can adversely affect project timing and significantly inflate project
costs.
Tip 4 – Collaborate with various business departments
The development and implementation of each DWH module need the active involvement of various business departments. Inadequate collaboration between these departments, a lack of understanding of the DWH significance, and excessive workloads often serve as
distractions that hinder the timely provision of data or the feedback required for acceptance testing. This issue affects all stakeholders and can result in launch delays, making it a matter that deserves the attention of everyone involved.
The end-to-end implementation of a data warehouse demands the active participation of numerous stakeholders in collecting requirements, defining the criteria for successful project outcomes, conducting acceptance testing, and providing user training. Therefore,
allocating time for these activities and establishing clear timelines is essential. A professional data warehousing vendor will communicate the expected time commitment from your end, which might involve dedicating a few hours per week to requirements gathering,
for example.
Tip 5 – Standardize data procedures and flows
If you plan to develop a data warehouse from scratch, it's evident that there might be issues with your data. The need for a centralized solution becomes apparent when you encounter various data-related challenges, such as incorrect data formats, inadequate
descriptions, missing data, disparate data formats across different departments, and more.
Here's how it may look like: your front office system fails to capture the last name of a lead, while your digital marketing system needs this name to be put in a specific field. The divergence in data formats across systems results in poor data quality
and the inability to make data-driven decisions.
Tip 6 – Conduct an audit of all information systems
In the realm of legacy banking systems, many institutions historically procured systems from external vendors. For instance, a bank might have acquired dedicated systems like the IBM zSeries mainframe, often termed monolithic systems, with a singular purpose,
such as supporting a deposit system. It was common practice for banks to employ a one-system-per-one-purpose approach in the past, creating vertical silos. These systems were constructed using various software generations, and the initial deployment often
occurred without consideration of integration.
Consequently, this approach gave rise to a complex, non-interoperable web of systems, making it exceedingly challenging to pinpoint data sources.
Tip 7 – Collaborate with executives
The success of any large-scale project within an organization hinges on active participation of executive leadership. In addition to fostering efficient inter-departmental cooperation, executives play a pivotal role in championing data warehouse projects
by elucidating the significance of data for the specific business domain under their purview.
Furthermore, C-level representatives from the business should articulate their needs, specifying the data they intend to supply to analytical systems, the data's originating sources, the decisions it will underpin, and a host of other critical details. This
collective commitment and guidance from the executive tier are paramount for project success.
Tip 8 – Follow Agile methodology
The Waterfall approach doesn't work for DWH projects. Instead, consider the Agile or Scrum methodologies, which enable project tasks to be tackled iteratively. In business intelligence initiatives, gathering and formalizing all requirements is nearly impossible.
Spending weeks with a business analyst soliciting user expectations for a system to be developed can result in time wasted, as users may not have a clear vision of their needs.
A more effective strategy involves incrementally seeking feedback on each release during sprints. This iterative approach facilitates prompt feedback collection and enhances the management of change requests.
Typically, we furnish customers with a comprehensive outline of the project management methodology related to data warehousing. This detailed overview is integral to the technical proposal, offering valuable insights into data warehouse implementation, project
timelines, the roadmap, communication plans, and team composition.
Tip 9 – Be ready for the change
In data warehousing, change is constantly driven by internal and external factors, such as evolving regulations or new data management policies. In anticipation of these changes, it is essential to ensure that your ETL (extract, transform, load) processes
and toolsets are designed with flexibility in mind.
Conduct thorough research to identify potential scenarios and proactively adapt your system to accommodate them. Change is an inherent part of the data warehousing landscape. While established best practices for data warehousing implementation can help mitigate
the challenges, you should be prepared.
Tip 10 – Opt for powerful memory devices
In data warehousing, it's vital to recognize that input/output operations often pose a bottleneck, especially for extensive queries. According to an Oracle report, modern hardware platforms offer a solution to reduce latency and eliminate input/output processes.
Memory devices such as DRAM (Dynamic Random-Access Memory) and Flash memory can significantly enhance system performance.
By comprehending the challenges inherent in data warehousing projects and adhering to the guidance above, you can potentially condense the project development timeline from several years to as little as nine-ten months. However, the exact timeline will also
hinge on the rigor with which you execute the project strategy. It is a critical factor in any IT endeavor, and particularly crucial in the context of a data warehousing project. Additionally, the successful implementation of these recommendations necessitates
close collaboration with a software vendor possessing robust expertise in data warehousing tailored to the needs of banking institutions.