Risk Governance in the Age of Artificial Intelligence

Accountability

Every AI system influencing a decision requires a named accountable owner who understands what it does, what it is used for, how it can fail and what happens when it does. Accountability arrangements that distribute responsibility across a committee, a vendor and a technology function tend in practice to locate it nowhere.

• Ownership sits with the business area whose decisions the system affects, not with the team that built it.
• Third-party and embedded AI capabilities require the same named ownership as internally developed systems.
• The board should receive an aggregate view of AI use, its material exposures and its control status, not a list of projects.

Inventory and scope

Most institutions have a model inventory built around quantitative models developed by identifiable teams. AI capabilities frequently enter through different routes: features within purchased software, general-purpose assistants used in analytical work, and tools adopted by business areas without central involvement. These fall outside the inventory by default.

1. Define scope by function — anything whose output influences a decision — rather than by technique or by who built it.
2. Run periodic discovery exercises rather than relying on self-registration alone.
3. Record for each entry: purpose, owner, data sources, validation status, known limitations, oversight arrangement and monitoring plan.
4. Include a materiality tier that drives the depth of control applied.
5. Reconcile the inventory against procurement and technology records to identify unregistered capabilities.

Data governance and human oversight

Data governance for AI extends beyond quality to provenance and permission: where the data came from, what it may lawfully be used for, whether personal data is involved, and whether third-party terms permit the intended use. For systems built on external content, licensing and confidentiality questions arise that conventional model governance did not need to address.

Human oversight is frequently documented and rarely effective. A reviewer presented with a recommendation, limited context and a processing target will approve nearly all of them. Effective oversight requires the reviewer to see the basis for the recommendation, to have time proportionate to the decision, and to face no penalty for disagreement. Override rates should be monitored as a control-effectiveness indicator.

Monitoring, auditability and generative systems

• Define monitoring metrics, thresholds and owners before deployment; retrofitted monitoring is rarely adequate.
• Monitor input drift, output distribution, override rates and outcome quality where outcomes are observable.
• Log the model version, input snapshot, output and any human intervention for every consequential decision.
• For generative systems, record prompts, retrieved context, model version and generated output, since reproducibility is otherwise unattainable.
• Establish a defined incident procedure covering detection, containment, fallback, notification and remediation.

Generative systems raise two specific governance problems. Non-determinism means the same input can produce different outputs, which breaks the reproducibility that conventional validation relies on. Fluency means output can be confidently wrong in a form that discourages scrutiny. Both argue for narrow, well-defined use cases with verification steps rather than broad deployment into consequential decisions.