Risk Matrix — AI Customer Assistant Risk Register (NIST AI RMF)
An AI risk register for a fictional utility's LLM billing assistant, mapped to NIST AI RMF and EU AI Act transparency duties: across nine failure modes from wrong tariff answers to prompt injection, controls cut exposure from 99 to 45 (−55%) and take the Critical prompt-injection risk to Medium, leaving missed vulnerable-customer handoffs as the one High risk above appetite. Illustrative register.
Make it your own.
title "Brightwater Energy — AI billing assistant, AI risk register"
subtitle "LLM assistant answering customer billing questions · mapped to NIST AI RMF and EU AI Act transparency duties"
likelihood-levels: Rare, Unlikely, Possible, Likely, Almost certain
impact-levels: Negligible, Minor (one customer, quickly fixed), Moderate (many customers or a complaint trend), Major (regulator interest), Severe (enforcement or mass harm)
bands: Low 1-3, Medium 4-6, High 8-12, Critical 15-25
appetite: Medium
note "Illustrative register for a fictional utility; failure modes, controls and ratings are examples, not an AI impact assessment."
risk AI-01 "Vulnerable customers are not handed to a human adviser"
category: Safety
owner: Head of Customer Care
likelihood: 3
impact: 4
control "Vulnerability classifier with one-click human handoff"
control "Daily review of conversations that mention debt or medical equipment"
residual: 2x4
action "Add domain triggers for life-support equipment and bereavement" due: 2027-03 owner: "AI Product Owner"
trend: falling
risk AI-02 "Assistant states a wrong tariff, refund or payment-plan policy"
category: Accuracy
owner: AI Product Owner
likelihood: 4
impact: 3
control "Answers grounded on the approved policy library, with citations"
control "Refusal when retrieval confidence is low"
control "Weekly human review of 500 sampled conversations"
residual: 2x3
risk AI-03 "Prompt injection exposes another customer's account data"
category: Security
owner: Head of Application Security
likelihood: 3
impact: 5
control "Tools scoped to the authenticated account only"
control "Output filter for personal data"
control "Prompt-injection red-team suite run in CI"
residual: 1x5
risk AI-04 "Vendor model update silently changes behaviour"
category: Robustness
owner: ML Engineering Lead
likelihood: 4
impact: 3
control "Pinned model versions"
control "1,200-conversation regression evaluation gates every upgrade"
residual: 2x3
risk AI-05 "Lower answer quality for Welsh speakers and non-native English"
category: Fairness
owner: AI Product Owner
likelihood: 3
impact: 3
control "Quarterly language-parity evaluation with a 5-point tolerance"
residual: 2x3
risk AI-06 "Assistant issues account credits without authority"
category: Agency
owner: Billing Operations Manager
likelihood: 3
impact: 4
control "Credits above £25 require human approval"
control "Daily reconciliation of every assistant-issued credit"
residual: 1x3
risk AI-07 "Prompts and transcripts retained beyond 30 days"
category: Privacy
owner: Data Protection Officer
likelihood: 3
impact: 4
control "Zero-retention agreement with the model provider"
control "Automatic transcript deletion and log redaction at 30 days"
residual: 1x4
risk AI-08 "Model provider outage during a bill-shock week"
category: Resilience
owner: Head of Digital Channels
likelihood: 3
impact: 3
control "Fallback to scripted self-service flows"
control "Second model provider on warm standby"
residual: 2x2
risk AI-09 "Customers are not told they are talking to an AI"
category: Transparency
owner: Data Protection Officer
likelihood: 2
impact: 3
control "Disclosure banner and a first-turn statement in every conversation"
residual: 1x3