Abstraction in Financial IT - How far can and should we go?

Abstraction is a fundamental concept in software engineering. An abstraction is a simplified version of something technical (with unwanted details omitted), which allows to hide away complexity via a simpler "interface" and is the main tool of a software engineer to cope with the ever-increasing complexity. Normally every programmer/user would need to resolve the complexity individually, but through the abstraction (creating a sort of black box in which the complexity is concealed) it is done for him.

It is therefore not surprising that in recent years we have added dozens of abstraction levels in the implementation of software products. These abstractions provide several advantages, like reduced complexity (easier and simpler), improved productivity (higher efficiency, easier reusability and reduced time to market) and a better encapsulation (increased inter-operability and easier replacement). With all those advantages, it is no wonder that abstraction has such a widespread usage.

Abstraction is everywhere in computer engineering, e.g.

• Computer programming abstractions:

  • Machine code: the ones and zeroes are an abstract representation of the electromagnetic signals flowing through the wires and computer chips.

  • Assembler: hides away the machine code, allowing a programmer to work with a more intuitive set of commands

  • Programming language: different programming languages exist with different levels of abstraction. These programming languages abstract away the complexities of an assembler, allowing a programmer to work with higher-level instructions (like e.g. for-loops and if-then-else conditions), which can represent dozens of assembler instructions. E.g. 1 line of Ruby code will be transformed by the compiler to a small novel in machine code. These abstractions make the code easier to read, simpler to write and much more efficient (a few commands give the same result as dozens of lines of a lower-level language).

    Picture below gives an overview of the abstraction levels of different programming languages (http://www.codecommit.com/blog/java/defining-high-mid-and-low-level-languages) with red being a low level of abstraction and green a high level of abstraction.

    

  • Within a programming language, some features like subroutines, modules, polymorphism…​ allow a programmer to create his own abstractions within the program.

  • Where a programming language typically allows to implement any type of application, there are also DSLs (Domain-specific languages), which provide further abstraction, but are also specialized to a particular application domain.

• Infrastructure abstraction:

  • Virtual machines: allow to abstract away a physical machine in order to create a number of virtual machines

  • Operating systems: abstract away all complexities of interacting with in- and outputs (like screen, keyboard, mouse, printer…​), memory management, disk and file management, networking…​

  • Docker containers: abstraction at the application level layer that packages code and its dependencies together.

  • Container management systems (Kubernetes): acts as a sort of operating system for distributed systems, allowing to abstract away a group of servers and manage them as 1 (cluster).

• Cloud provider abstraction: cloud providers are continuously adding abstraction levels, making the architecture more and more "cloudy":

  • Hardware abstraction: cloud providers first started to abstract away the underlying hardware resources, allowing users to pay-as-you-go for services such as storage, networking, and virtualization, i.e. Infrastructure as a Service (IaaS).

  • Managed services: gradually cloud providers moved the abstraction levels up to managed databases (like AWS Aurora, RDS or Dynamo DB, Azure’s Cosmos DB or Google’s Bigtable), managed monitoring and logging solutions (e.g. Amazon CloudWatch), managed container management systems (like Amazon EKS, Google Cloud Kubernetes Engine or Azure Kubernetes Service), pre-loaded application software like web servers or CRM solutions…​

  • Serverless: allows to abstract away servers completely, making functions run as a service (i.e. hardware resources are only attributed at the moment of the execution of the function), e.g. AWS Lambda, Google Cloud Functions, Azure Functions or IBM OpenWhisk

  • Specialized abstractions, like Machine Learning abstractions (e.g. GCP TensorFlow or Azure Machine Learning), sending communications like SMS’es and mails (Amazon SNS), CI/CD pipelines (e.g. AWS CodeCommit, CodePipeline, CodeStar and CodeDeploy) or even abstracting away manual tasks, i.e. crowdsourced labor (via Amazon Mechanical Turk).

• Solution architecture abstraction: within a solution architecture, software engineers also try to componentize the application landscape in smaller (and hopefully simpler) blocks, which are a maximum encapsulated from each other. This gave rise to different architectural principles, like ESBs and SOAP and more recently to the micro-service based architecture. In this architecture, processes are encapsulated into a micro-service, which is a black-box and communicates to the outside world via a well-defined and usually simple interface, called an API.

• …​

Typically, every new abstraction which is created, gives rise to the creation of new abstractions to hide the new complexities introduced by the new abstraction layer. E.g. the rise of Kubernetes has resulted in dozens of new abstractions like pre-defined Kubernetes setups (like Kops, Bootkube…​), serverless setups based on Kubernetes (like Kubeless, Knative, Fission), shells and dashboards, service meshes (abstract away complexity of services communicating which each other, i.e. security, routing, monitoring, tracing…​)…​ Each of these tools build a layer of abstraction on top of Kubernetes, providing a simplified "interface" on a complex feature of Kubernetes.

We could even say that every application provides some kind of abstraction layer on top of existing abstraction layers. The difference is that a business application will be on the top of the abstractions, while infrastructure tools are situated at lower levels and are therefore also used by multiple business applications. For example: you can build a website by hardcoding the HTML, but you can also use a CMS (Content Management System) to manage the content of the pages. The CMS in itself is however not a business application, but the website created with the CMS can be.

But even higher up the stack, abstraction can be a useful tool. Business processes and products can also gain from abstractions, as they facilitate faster innovation and easier collaboration. Banks can abstract away their complex back-office processes and complex product characteristics via easy APIs and product factories, allowing easier creation of new products and adaptation of existing business processes. They can also use external abstractions offered by SaaS solution and Fintechs to rapidly provide new user experiences.

With abstraction levels built one on top of the other, you can easily ask yourself, where does it end? The ultimate goal of course is that any person with a business idea, but no IT background at all, can built a complex IT system, meeting the strictest non-functional requirements. Low-code platforms like Mendix, Outsystems, Kony Quantum, Microsoft PowerApps…​ provide a step in this direction, but there is still a very long way to go, which brings us to the downsides of continuously adding levels of abstraction.

With each abstraction comes also opinions, assumptions and restrictions (i.e. enforced scoping constraints). Typically, the developer of an abstraction layer will create an opinionated system, i.e. based on several best practices he will implement the specifics on the lower level. The result is that someone using the higher abstraction level has automatically lower flexibility. If your usage fits well the applied best practices and constraints, the abstraction level is ideal, but if not, the abstraction level comes at a significant cost. Adding an abstraction is therefore always a compromise (i.e. you gain speed, but lose flexibility), meaning there is an optimal level of abstraction, which is dependent on the end goal.

With each abstraction, people also lose insights. When learning a new topic, it is important to continuously zoom-in and out on the subject to explain and understand the subject at different levels. When someone explains a topic only at the lowest level, you will lose yourself in the details. However, if the message remains too abstract, the subject will feel fluffy and not very concrete. Due to the increased level of abstractions, people tend to think about system only at a higher-level level, which will pose issues at a certain moment. This is where the law of leaky abstractions comes in. Any abstraction will ultimately leak, meaning that at some point something will go wrong and you will only be able to fix it by digging into the lower level operations.

As the above demonstrates, abstractions are a powerful tool, when used consciously. In banking, unfortunately the modern abstractions are not very well introduced yet. Banks are still built on top of legacy software implemented with lower-level programming languages like Cobol, very little to no usage of cloud solutions and still very monolithic, not very-well encapsulated architectures. Furthermore, from a business point of view, the products and services are organised as silos with little to no abstractions.

As indicated above, any technological evolution comes with new abstraction opportunities. With the banking sector being in such fast evolution the last few years, opportunities for abstractions are everywhere. Companies that can introduce the (technological) abstractions of modern technology stacks in a controlled and efficient way in existing large banks will have massive opportunities in the coming years. E.g. core banking transformations, (partial) migrations to the cloud or gradual conversions of the bank monoliths to a microservices based architecture.
But there are also opportunities in more specific banking-oriented abstractions. Every bank topic or service, which is complex to understand or use for employees and/or customers, gives an opportunity for an abstraction layer. E.g. complex processes like investing, credit risk management, fraud detection, hedging…​ should also be abstracted away, so that the end user can express his desired outcome rather than having to understand all the details of the underlying processes.