One of the main complexities in AL/ML lies in understanding and explaining the predictions made by these models. While they outperform simpler expert rules or statistical inferences, they often operate as black boxes, making it difficult to grasp the decision-making processes behind them. This paper provides an overview of additional requirements for model validation in the context of AI/ML.
Validating any model starts with ensuring sufficient documentation and testing to evaluate its quality and appropriateness. Once documentation is assessed, evaluating the strength of model inputs becomes crucial. AI/ML models rely on statistical analysis and generalisation from observed patterns in the calibration set to make future inferences.
Whilst decent data sourcing and feature choice are a pre-requisite for model quality, the choice of core AI/ML inference model for analysis can lead to material difference in the information extraction. This is especially true where transfer learning is used for critical information processing, e.g. to tokenise or convert features ahead of the main inference.
Contents
- Bigger Data, Bigger Problems
- Documentation Provides the Foundation
- Feature Engineering Drives Performance
- Core Inference Model Selection
- Enhancing Model Governance
