Beyond explainability: A practical guide to managing risk in machine learning models

22 June 2018

This report offers a comprehensive guide for effectively managing risk in machine learning models. It presents a framework that enables data science and compliance teams to create better, more accurate, and more compliant models. The report stresses the importance of understanding the data used by models and implementing three lines of defence to assess and ensure their safety.

Please login or join for free to read more.

OVERVIEW

This report presents a framework for effectively managing risk in machine learning (ML) models. As investments in the US alone are expected to approach $8 trillion in the coming years, the complexities of ML make for growing legal, ethical and reputational challenges. The report offers a practical guide for data-driven organisations, data scientists and legal personnel, which includes three lines of defence to ensure the safety and security of ML models over time.

Focusing on the input data, one of the most critical yet overlooked aspects of governing ML, the report provides recommendations on effectively managing the data infrastructure and training datasets as well as documenting model requirements. It highlights that proper roles and processes must be put in place to direct multiple tiers of personnel to assess ML models. To that end, the report recommends setting clear objectives and expected outcomes before deploying the model and specifying expected consumers of the model.

The report highlights the importance of documenting all analysis and testing results and addressing all potential risks associated with ML’s deployment. It stresses that every model has unforeseen risks, and recommends that the depth, intensity and frequency of review factor in aspects like the model’s intended use, restrictions on its use, its potential impact on individual rights, the quality of the model and the data, and the level of explainability. The report emphasises that the quality of the process is judged by the extent and clarity of documentation, the issues identified by objective parties, and the actions taken by management to address the issues.

Recommendations

Set clear objectives and expected outcomes before deploying ML models
Specify expected consumers of the models from individuals to systems
Implement three lines of defense to assess the safety and security of the models over time
Focus on the input data, train datasets and document model requirements
Document all analysis and testing results and address all potential risks associated with ML deployment
Tailor third line reviews to the specific risks of the ML in deployment and to the specific compliance burden as well.
Factor in aspects like the model’s intended use, restrictions on its use, its potential impact on individual rights, the quality of the model and the data, and the level of explainability for setting the depth, intensity and frequency of the review.