Robust and Defensible

OVERVIEW

In this post, we will consider how Artificial Intelligence (AI), specifically Natural Language Generation, can ensure robust and defensible decision making processes that suffice regulatory scrutiny.

IT'S NOT THE DECISION.  IT'S THE PROCESS

“Robust and Defensible” was a common theme at a recent JP Morgan conference that examined the challenges faced when providing accurate, market-based, and independent valuations of securities. This theme is applicable across the entire regulatory and compliance landscape.

When auditors come in to evaluate your operations, it is not the final decisions or opinions that matter, but how they were determined. What was the process? Is that process well documented? More importantly, was the process followed?

HAVE A GOOD CONVERSATION

One panelist at the conference stated that he would rather have a debate about an off-market price than have a long drawn out conversation about an ambiguously documented decision making process. A worse conversion would be about a clearly documented process that was not followed by a member of the team.

A good conversation highlights a sound decision that is based on a clear and adhered to methodology, not the actual decision.

AUTOMATE THE RIGHT WORDS

An opinion is communicated through words. After the data is collected and the analytical machines have run their algorithms, an analyst (person) applies their expertise to understand the resulting data set (graphs, charts, and tables), formulate an opinion, write it down, and publish.

The problem lies in keeping up with the continual ask for transparency from regulators and clients and the tremendous growth in data being analysed. The expert (the person) can’t efficiently keep up and the quality of their opinion may vary from day to day.

A subset of AI, called Natural Language Generation can generate words automatically, but they need to be the right words. There are many providers of this capability. When evaluating a solution for regulatory scenarios, there are a few things to consider.

CONTROL THE LANGUAGE

Regulatory responsive language should be generated from scratch, in a controlled fashion. The AI should create the narrative only from business rules (your codified expert) and the data. A solution that constructs sentences from pre-existing phrases, no matter how efficiently and well-structured, is only a statistical best guess and not fully defensible.

BE CLEAR AND CONCISE

It is easier to write a lot of detail than to write clearly and concisely. The ability to summarize concepts from detailed data is a real skill, both for man and machine.Make sure the solution is able to tell the story in a few sentences, not multiple paragraphs.

GIVE THE REGULATORS WHAT THEY WANT

By controlling and effectively communicating the language (codifying you best analysts) you are giving the regulators what they want in an automated way. The decision making process is auditable and the output always follows the rules. The process is robust and defensible.

Link here and here for this and other related posts.