Navigating Model Risk Management Regulation On A Global Scale

By Miles Elliott, Risk Management Advisory Lead for EMEA at SAS, and David Asermely, Global Lead for AI Governance and Model Risk Management at SAS.

Regulators around the world differ in their approach to model risk management (MRM) regulation – including their definitions of what a model is. While some are more prescriptive, others such as the UK, have deliberately opted for a broader definition.

In this blog we will explore some of the key elements of MRM regulation across three territories - the US, the UK and mainland Europe. 

But first, where are firms expending their effort today?

Variation in regulatory guidance across different jurisdictions has required firms to expend more time than perhaps expected on defining what a model actually is.

Firms are reviewing how best to govern model risk, be this in relation to how they address proportionality across geographies, within business units or between models.

Some firms are exploring how they should vary model risk policy by domain noting that some domain areas incur greater inherent model risk driven by for example data limitations, or methodology limitations.

Whilst other firms are exploring how they should reflect operational risk driven by the complexity of the associated business process and the degree of manual intervention, e.g. in relation to manual adjustments.

The question of how firms should position the specific role of the model risk management function is also being explored, be that in terms of how the function should shape overall model risk management & governance, or what role the function should take in the post model adjustments process.

Still other firms are bringing things back to basics by challenging themselves to have a comprehensive and transparent model inventory, coupled with flexible validation and monitoring policies, that is easy to manage and that enables them to determine if and where model risk is present.

…and all of this is driving firms to explore solutions that can support them accelerate their MRM capabilities, so as to better identify, manage, govern and mitigate model risk.

Regulation in the US

As we have seen in an earlier byline the US led the way with SR11-7, thereby setting the standard around the world. Despite the seismic changes that have taken place in the world since then, feedback from the US financial services industry suggests that the regulation remains relevant. 

That said, US guidance has continued to keep pace with market changes and emerging technologies. 

US regulators appear to be mindful of the interconnectedness of model risk from the perspective of whether certain modelling aspects - e.g. those linked to interest rate correlations - generate adverse model risk not only within a discrete set of models but more widely, generating a systemic impact.

When conducting sensitivity analysis, firms are urged to take learning from Covid-19 - where models built before these periods did not hold up well during the period thus requiring remediation and post model adjustment - and applying it to the currently higher inflation rate environment to understand the impact of model risk on resulting model outcomes.  

US regulators suggest that model risk goes far beyond the model itself – identifying that data management and data governance is increasingly important given the additional volume \ types of data, for example coming from AI.  

Equally, they seem to appreciate that AI practices should be evaluated on a case-by-case basis to determine whether they should or should not be included in a model risk management framework by considering to what extent such practices actually introduce incremental model risk.

US regulators seem to recognise that treating customers fairly and appropriately must be treated carefully irrespective of the technology used, e.g. AI. Here we can draw a parallel to some extent with what we see in the UK, through the FCA’s regulation on Consumer Duty and their discussion paper (DP5/22) on AI & Machine Learning.

As cited in a previous blog, climate risk was one of the drivers behind the publication of PS6/22 in the UK. It seems US regulators are also recognising the additional challenge here for model risk, noting the volume of data (both too much and too little), model complexity (by incorporating physical risks into models) and the availability of benchmarks (which can show very different views).

Regulation in the UK

As mentioned previously, UK regulators seem to have intentionally kept their definition of a model broad to ensure MRM frameworks would incorporate the wide range of different model types in use. 

Nevertheless, it has clarified that the definition of a model extends beyond traditional ‘risk’ models and that ‘examples of highly complex quantitative calculation systems that could have a material bearing on a firm's financial position include Electronic Trading Systems that are made up of a complex interdependent network of components, and which may constitute a model, as well as financial crime and / or anti-money laundering systems’.

Further, with dedicated principles associated with governance, UK regulators require firms to exhibit greater accountability regarding model development, implementation and use; have clarified that vendor models are not required to disclose proprietary information; and from the perspective of model mitigants careful review of post model adjustments is required.

Regulation in mainland Europe

Given the EU will soon adopt its AI Act, risk experts might be wondering what that means for MRM. 

The Act sets out requirements around data management & data governance, documentation & traceability, bias management, accountability & oversight, transparency throughout the model lifecycle and includes overarching expectations for the design, execution and governance of high risk AI systems.  The Act requires the classification of AI systems according to four levels of risk – unacceptable, high, limited or minimal/low. Interestingly from a model risk perspective, it is currently proposed that the list of high-risk AI systems may extend to include the assessment of a customer’s credit worthiness or associated credit score as well as decision making in relation to health and life insurance. 

…and for firms operating internationally?

Variability we have seen in the regulatory landscape across jurisdictions means that firms’ MRM practices will likely also vary across these same jurisdictions.

With the publication of SS1/23 it is reasonable to expect that firms will be challenging themselves to further drive out some of this variability.

We can expect that firms will explore moving to a harmonised definition of a model and its associated materiality to better enable attestation at a global level – and directly linked to this we can expect firms to have increased appetite to adopt a common model inventory.

It is also reasonable to expect that firms will want to drive greater standardisation in business processes across the model lifecycle to assist with the roll out of firm-wide model risk policy – and the ability to monitor it.

And with analytics capabilities advancing at pace it is likely that firms will search for additional ways to drive out increased process efficiency – by integrating technology and furthering automation.