Model Governance in an Open-Source World
Part 1: Design and Implementation of a Successful Model Governance Framework
The model governance framework is a structured set of protocols that govern the use of modeling tools and provide guidance for the use of those tools. The initial steps in setting up a framework are to set up the environment, define the scope and establish the role of the model steward. The next three sections provide some initial guidance to define these three areas.
Setting the Environment
The implementation and preservation of a governance framework requires unwavering corporate commitment and a virtuous risk management culture. A newly formed governance process needs a lot of care and commitment for its processes and rules to be established and followed. As with any new infrastructure, there will be those resistant to change and governance. Therefore, it is important to enlighten all affected parties about the purpose and importance of the new framework. It will also be beneficial to involve all potentially affected parties in the development of the new framework and its processes. People tend to support structures they helped develop and implement. A risk management culture needs to be woven into the larger operating DNA of the department and company at large.
Defining the Scope
The ultimate scope will include all models that functionally impact the company. However, at the onset of implementing a governance process, it would be wise to select a few key models to subject to the new framework. The new framework will invariably require adjustments to address unique processing components and inefficiencies, and to avoid redundancies. Once the governance framework is sufficiently perfected it is ready for additional scope. A majority of these functional processes include:
- Data transfer
- Model enhancements
- Model validation
- Model corrections
- Archival of models
- Model results usage
- Management approval
- Software upgrades/conversions
- Peer review
Establishing the Function of the Model Steward
The primary responsibility of the model steward is to make sure the instituted model governance framework processes and controls are followed. The steward may not be appreciated at first by the model developers and users, and the model users and developers will have pain points as the rules are enforced. Some of the early pain points experienced may be:
- Protocol prevents making changes on-the-fly
- Perception that management no longer trusts capabilities of users and developers
- Increase in meetings, write-ups and analysis rather than making model code changes
- Too much time spent on documentation
As time goes by and governance processes are fine-tuned with the help of the model steward, developers and users will realize the framework is effective in reducing model risk.