Selecting Right Database for right workload

Enterprises are nowadays working on modernize and trying to come out of legacy systems which are hard to maintain. Time consuming in terms of ready to market. Some are not that customer centric. Hence while working on the modernization of technology stack, one often asked question would be: What could be my target database engine? Where should my application be hosted? Where should be my Databases hosted? With all of these I need to address my RPO/RTO requirements.

There is no thumb rule to address to pick the right database for right type of use case. As there are hundreds are options available with all their own pro and cons. We need to diligently select the databases most probably pick two or three that meets your criteria.

Before shortlisting any specific databases, have a deep understanding of the business requirements to determine the application functionalities and then the type of database application that might need to be implemented. The biggest question about implementation is, should we go with "SQL or NoSQL" and "On-premises or Cloud"

Most of the time business demands cost savings to meet the TCO. There are lot of options available which does not demand the licensing and most of the enterprises gets attracted to such products. Cost saving can impact the overall project when it comes to architecture, performance and availability. These products come with different capabilities to address the real-world issue. Each of these products offers various range of solutions hence it becomes difficult to pick the one that suits one’s need.

Business Driver for Data-Strategy 

As this is enterprise-wide change there must be some business motive behind this exercise, it could be either one or multiple for the below: 

• Improve IT infrastructure and reduce data-related costs (number of databases, etc.)
• Generate return on investments in big data and analytics infrastructure
• Leverage new sources of data, internal or external
• Improve the quality of data
• Rationalize multiple sources of the same data and information
• Monetize the value of the company's data; use internal data as a product or service
• Meet industry regulatory requirements
• Optimize existing strong bench of analysts and data scientists
• Prevent cyber attacks and data breaches
• Create new products and services
• Develop analytics and digital capabilities
• Reduce general operating expenses and streamline business processes
• Use sophisticated, real-time or near real-time analytics for business
• Mitigate operational risks such as data breaks, fraud, etc.
• Improve revenue through cross-sell, pricing, and expanded customer base
• Respond rapidly to competitors and market changes

To address the business requirement, one must know what’s there in the current set-up. For this careful approach need to be chosen to deep dive and understand. Identify the duplicates and try to eliminate. The care should be taken to get rid of the unwanted / outdated to be scrapped or replaced. Some COTS to be reviewed to make sure the latest and better solution available in the market to replace those. Database migration is one aspect to look at but we must not ignore the applications it is connecting to. Hence this approach should be looked in to application point of view. 

Application analysis:

• Collect all the application with their technology stack
• Understand the nature of application
• Group them on the nature of business and the Users (inhouse or external)
• Scaling requirement
• Business criticality
• Size in terms of DB size, transaction volumes, Transaction type (Read/Write), number of users connecting simultaneously, User Geography
• Are you planning for applications to be DB agnostic in the future state

ETC.

Select a couple of databases in the chosen database category:

• Map each application to a relational database (SQL) or NoSQL categories.
• Select two databases based on the mapping.
• Carry out in-depth study of the chosen Databases to ensure it meets your application demands.
• Tool availability to migrate to target DB and Application
• Determine the timeline for entire program

Consider following before starting the database evaluation process.

What would be your hosting plan?

On-premises — you would provision and manage everything on your data center — the host machines, the operating system, and the database cluster.

Cloud-hosted — a cloud provider (e.g. AWS, GCP, Azure) would provision and manage the host machines, whereas you would manage the operating system and the database cluster on those cloud-hosted machines.

Managed database as a service — a database service provider would provide and manage everything for you. E.g. DynamoDB by AWS. Based on the requirements, both parties (you and the service provider) agree on an SLA and the necessary cost is charged by the provider based on the usage. Be aware of the vendor locking with some solutions, which means, migrating back to On-premises or Cloud hosted approaches in the future might not be straightforward.

Would you like to go with the Enterprise or Community edition?

• Enterprise edition — paid software version with advanced features and support.
• Community edition — free software version with basic features and no support.

Criteria for choosing a database:

• Database procurement staff pay more attention to purchase costs, including storage and network requirements.
• Database administrators (DBAs) care about:
  • Operation and maintenance costs:
    • A reliable monitoring and alerting system
    • Support for backup and restore
    • Reasonable upgrade and migration costs
    • An active support community
    • Ease of performance tuning
    • Ease of troubleshooting
  • Service stability:
    • Support for multiple data replicas
    • Highly available services
    • Support for multiple writes and multi-active architecture
  • Performance:
    • Latency
    • Queries per second (QPS)
    • Whether it supports more advanced hierarchical storage features
  • Scalability: Whether it’s easy to scale horizontally and vertically
  • Security: Whether it meets audit requirements and prevents SQL injections and information leakage
• Application developers care about:
  • Stable services
  • Performance
  • Scalability
  • Ease of developing database interface
  • Ease of modifying the database schema

Conclusion:

Process of selecting right database need to crafted carefully. Organization should work on decision making tree to address core business need based on the above points.