20 min
Lesson 3 of 4
Risk Monitoring and Reporting
From the course AI Risk Management
Establish ongoing monitoring programs and reporting cadences for AI risk management.
Monitoring Program
Automated Risk Indicators
class AIRiskMonitor:
thresholds = {
"accuracy_degradation": 0.05, # 5% drop triggers alert
"bias_disparate_impact": 0.8, # Below 0.8 is high risk
"error_rate": 0.001, # 0.1% error rate threshold
"data_drift_score": 0.1 # KS statistic threshold
}
def check_risks(self, model_id, metrics):
alerts = []
if metrics["accuracy"] < self.baseline[model_id] - self.thresholds["accuracy_degradation"]:
alerts.append({"type": "ACCURACY_DRIFT", "severity": "HIGH"})
if metrics["disparate_impact"] < self.thresholds["bias_disparate_impact"]:
alerts.append({"type": "BIAS_DETECTED", "severity": "CRITICAL"})
if metrics["drift_score"] > self.thresholds["data_drift_score"]:
alerts.append({"type": "DATA_DRIFT", "severity": "MEDIUM"})
return alerts
Risk Reporting
Risk Register Template
| Risk ID | Description | Likelihood | Impact | Score | Owner | Controls | Status |
|---|---|---|---|---|---|---|---|
| AI-001 | Model bias in lending | 3 | 5 | 15 | Head of Risk | Bias audit, human review | Active |
| AI-002 | Data poisoning | 2 | 4 | 8 | CISO | Input validation, training data scanning | Active |
Board-Level Reporting
- Quarterly AI risk dashboard for executive team
- Annual AI risk report for board presentation
- Incident reports for material AI failures
- Regulatory filing support for compliance matters
Key Risk Indicators (KRIs)
- Model accuracy vs baseline (weekly)
- Bias audit results (quarterly)
- AI incident count and severity (monthly)
- Regulatory inquiry count (quarterly)
- Staff AI ethics training completion (quarterly)
Key Takeaways
- •Automated monitoring with threshold-based alerting catches risks before they escalate
- •Risk registers provide structured tracking of identified risks and their controls
- •Board-level reporting translates technical risks into business impact language
- •Key Risk Indicators provide leading indicators of emerging AI risk
0 of 4 lessons marked complete
Progress is kept in this browser only.