Third party model validation Third party model validation

Recent supervisory activities and regulatory guidance indicate that third party model validation is increasingly becoming an area of concern to European supervisors, requiring banks to comply with heightened expectations. Banks needs to ensure they have adequate in-house knowledge, and to remember that they remain responsible for all activities related to their internal models, including overseeing all risks and managing the outsourcing arrangements.

In its pursuit to ensure a consistent approach to internal models, the European Central Bank (ECB) recently published a revised version of the ECB guide to internal models - General topics chapter1 . The revised guide reinforces supervisory expectations in relation to non-model specific topics, including the measures which should be considered by institutions where a third party service provider is involved in any tasks relating to the rating systems, such as model validation activities.

We have previously discussed how the ECB's Targeted Review of Internal Models (TRIM) has raised expectations around standards of model risk management and so this scrutiny should come as no surprise. As one of the supervisory priorities for 2019, TRIM continues to remain an important project for both the ECB and national competent authorities. The preliminary results of TRIM investigations, disclosed by the ECB last year, highlighted numerous weaknesses around validation activities such as inadequate resources allocated to the internal validation function, and inadequate separation of staff performing validation activities and model development.

Banks often have insufficient in-house capabilities to internally manage model validation activities mainly due to a lack of skilled resources and inadequate IT infrastructure. Therefore, financial institutions may choose to involve a third party service provider to perform model validation tasks, enabling them to focus on their core business and giving them access to tools and services that are not available in-house. However, it's vital that those third parties validating banks' models understand the related model risks and have the resources, expertise and independence to perform thorough assessments.

The ECB guide to internal models identifies regular validation of models and appropriate data quality practices as essential elements of effective model governance. Distorted models have growing potential to misconstrue otherwise solid lending, trading, balance sheet management or financial reporting, thereby having an adverse impact on the strategic decision-making of banks - such as business decisions related to capital calculations or liquidity. Under these circumstances, it's important that third parties involved in the validation of banks' models are able to carry out effective, unbiased assessments and that they meet supervisory expectations at all times.

In order to maintain sufficient in-house understanding of validation, the ECB considers it `advisable' that banks should:

• Understand major model assumptions and risk estimation processes; 
• Retain access to information provided by borrowers;
• Compare any external data used in the validation process with their own;
• Understand what definitions of default are being used;
• Retain access to all the data needed to perform independent validations;
• Establish a change policy covering models developed by third parties; and
  
• Agree the triggers for, and the processes of, model change with service providers.

The ECB also suggests that good practice in terms of third party oversight should include:

• Performing similar data checks as if validation were occurring in-house;
• Investigating any data anomalies;
• Assessing the suitability of any external data being used;
• Cross-checking data consistency between databases and providers; and
• Including specific metrics and KPIs in any contracts or Service Level Agreements. 

In addition, the recently published EBA's revised guidelines on outsourcing of February 2019 re-emphasised the responsibility of management bodies for all of their institutions' activities, and reminds banks that they must ensure sufficient resources are in place to oversee all risks and outsourced activities.

These new sets of guidelines and supervisory initiatives suggest that many banks have work to do to ensure they're meeting European authorities' expectations. At a minimum, we believe banks should:

• Establish robust model governance frameworks, to ensure compliance with guidelines,
• Proactively identify and mitigate potential model issues, as well as limitations, before involving a third party for model validation tasks;
• Develop and maintain sufficient in-house knowledge to oversee outsourced model validation effectively; and
• Ensure that they have robust processes, procedures and supporting documents to facilitate adequate understanding of the model by third parties. It is equally important for banks to transfer appropriate knowledge to the third party in order to perform effective validations. For instance;
  • How models source, use, and transform data, and how banks address missing or incompatible values;
  • Where banks use external models, third parties must be supplied with adequate technical information so as to understand how externally developed models have been set-up, and how they have been customised by banks including sufficient details on any judgemental parameters.

Supervisory focus on model validation by third parties is expected to gain traction in the near future. This may primarily be a technical issue, but it's one with real implications on the strategic decision making of banks. It also has the potential to evolve further as TRIM investigations continue and new findings are raised. Banks must not allow it to become an area of weakness.