Model risk management (MRM) allows complex models to be used with greater confidence, identifies model enhancements, mitigates model weaknesses and demonstrates sufficient internal controls to a firm’s leadership and external stakeholders. This blog provides an overview of the initial questions that firms need to answer when considering a firmwide approach to model governance.
Does my firm need a separate model validation function?
Not necessarily. The complexity of MRM needs to be tailored to the firm’s current situation. A rigorous and fully comprehensive model validation process can be resource-intensive as it requires investment in personnel, technology and training. Adding validation steps as prerequisites to deploying new models in production, without sufficient resourcing to review in a timely manner, would slow a firm’s ability to respond to changing client demand or market conditions. A model governance function may suffice for a financial firm when regulatory requirements are not prescriptive and the number and complexity of models is low.
With a robust MRM process in place, a firm benefits from improvements in model risk awareness and its ability to rely on models for business decision-making.
Model validation can be thought of as a natural evolution of model governance as the complexity and number of models increases. As a firm grows or its models become more complicated, it can be difficult for model governance functions to dedicate time from their business-as-usual risk assessments to perform analysis of the methodology, code base and proposed operational implementation of new models or large-scale modifications. The model governance function needs to collaborate with developers and users; a more distant model reviewer can potentially provide a more independent (that is, potentially negative) opinion on model suitability. Financial firms therefore develop a model validation function to enable the detailed and rigorous analysis of the methodology, mathematics and implementation of models.
Model validators take a collection of information – documentation, tests, code – to compare model suitability against a described usage scope. A formal validation process ensures that a model meets a minimum set of specified standards. A typical review results in approval or disapproval, potentially with mandated enhancements or requirements for further investigation. By scheduling validation reviews, firms can identify issues and build in remedial development work.
How does MRM protect profitability and reputation?
Model risk management seeks to mitigate direct losses associated with model flaws, such as misquotes in market making, and mitigate the mistakes in risk management that could occur from inaccurate metrics, such as poor market or counterparty credit risk estimates. As well as controlling financial losses and opportunity costs from bad decisions, MRM helps prevent reputational damage.
With a robust MRM process in place, a firm benefits from improvements in model risk awareness and its ability to rely on models for business decision-making. The subsequent advantages include:
- Increased confidence in model outputs
By demonstrating due diligence in model risk management, firms can increase shareholder, regulator, investor and customer confidence. Models developed for new clients or products are validated to prevent problems occurring. - Better accuracy for models by identifying enhancements
MRM can help identify and correct errors or limitations in models, leading to improved accuracy and reliability. Even if a model has been well selected, effective MRM may highlight opportunities for further improvement. - Superior decisions using imperfect metrics
By having a better institutional understanding of the strengths and weaknesses of models, MRM can help ensure that decisions based on flawed but useful models are well informed and more likely to achieve desired outcomes. - Compliance with regulators and counterparty expectations
To avoid penalties or reputational damage, MRM can ensure that models are compliant with relevant regulations and industry standards. Some central clearing entities such as exchanges require evidence of adequate internal model controls. Strong internal controls can also support better credit ratings and reduce borrowing costs.
Ensuring service level provision and code robustness can be difficult if the third party does not commit to the levels required internally. Licenses need to be well constructed to reduce uncertainty over future costs and permit model extensions to new potential usages.
How are third parties able to support MRM activities?
When implementing an MRM policy, a core question that arises is what external assistance is required to support governance and validation.
Support typically fits into four categories:
- Direct use of third-party model tools to perform calculations rather than in-house software
- Use of third-party tools for governance, such as model inventory software
- Use of third-party tools for validation, such as calling external APIs to benchmark pricing, risk and credit evaluation models
- Use of consultants to carry out validation activities or to verify the quality of internal validation activities
Third-party support solutions can simplify overall model risk management. With direct use of third-party models, firms benefit from:
- Prior analysis and verification activity by the third party
- Improvements based on client usage feedback
- An ‘oven-ready’ supply of documentation and testing
- Up-to-date codebase with innovations in market conventions and best practice
- Faster reconciliation than independent reimplementation
There are, however, challenges with replacing internal model with third-party solutions. There can be a lack of transparency where third parties do not fully document calculation details. Ensuring service level provision and code robustness can be difficult if the third party does not commit to the levels required internally. Licenses need to be well constructed to reduce uncertainty over future costs and permit model extensions to new potential usages. In addition, where models need to be specific to the firm, or contain IP advantages over competition, then standard third-party models may be unsuitable and custom software necessary.
These negative issues are considerably reduced if the intended use of the third-party models is for validation against internal models rather than full replacement. Consistent results can evidence model accuracy even if some calculation aspects are unclear. Use of standard models for benchmarking helps ensure alignment with common market practice. The choice of model provider for validation does not create ‘lock in’ – usage is time-restricted and a different vendor’s models could be used for validation two years after an analysis is completed, making reliance lower and cost assessment simpler. Service-level provision is less critical for periodic validation.
We can therefore consider validation being suited to third-party models, even for entities that wish to retain full control of their models’ production code base. In some cases, additionally outsourcing the model calculations may make sense to reduce internal development, maintenance and validation costs, where the third-party provider can provide licence flexibility and evidence of model reliability.
Conclusion
MRM seeks to establish the acceptable model risk appetite mandated by a firm’s leadership and percolate that requirement down to restrictions around model development and usage. To control model risk a firm can establish a formal model governance policy and procedure around model identification, tracking and usage restrictions. As the scope and complexity of model usage grows, there are further benefits from establishing a dedicated model review function. Third-party assistance may reduce the cost and complexity of MRM where external models are suitable for direct use or validation. In summary, establishing a formal MRM process creates additional costs but mitigates potentially expensive risks. MRM allows complex models to be used with greater confidence, identifies model enhancements, mitigates model weaknesses and demonstrates sufficient internal controls to a firm’s leadership and external stakeholders.
