Responsible AI assurance: from governance documents to proof that controls work cover

Responsible AI assurance: from governance documents to proof that controls work

Listen to this article

​A practice guide for UK financial services by T3 Consultants October 2026

The question is no longer “do you have an AI policy?” It is “can you show your safeguards work in this use case?”

The FCA’s message in one page

Six decisions every firm must be able to defend

  1. Purpose: Why AI fits the need — and when it must abstain or escalate.
  2. Ownership: Who owns it, and who can challenge, restrict or suspend it.
  3. Testing: Which harms, error patterns and dependencies were tested before launch.
  4. Thresholds: Which metrics signal success or harm, and what triggers intervention.
  5. Oversight: Whether human review is genuine, not a formal approval step.
  6. Change: What happens when models, prompts, suppliers or data change.

Three tests of assurance

  1. Design: Are controls proportionate to the use case and its harms?
  2. Operation: Is there evidence approvals, monitoring and escalation happen as designed?
  3. Effectiveness: Do repeatable tests show controls prevent or catch material failures?

What the publication is — and isn’t

Illustrative, non-exhaustive examples (19 Oct 2026, updated 20 Oct). It creates no new requirements and no prescribed governance model. Binding obligations still come from existing rules. [FCA §1.2]

Next 30 days

  1. Validate the AI inventory against reality.
  2. Pick two material use cases for evidence-based review.
  3. Put a live exception dashboard beside the policy pack.

A missing artefact is not automatically a breach. Test applicability, actual arrangements and outcomes.

Context and method

An evidence base, not a survey

The FCA drew on engagement with firms, industry and consumer bodies, charities, policymakers, technology providers and academics — plus AI Live Testing, the Supercharged Sandbox, the “AI in smaller firms” event, the AI Input Zone and Mills Review work. It applies to any regulated firm building, buying or using AI, including embedded and third-party tools.

Not disclosed: how many firms took part, who they were, or the size and sector mix. Case studies illustrate practice; they don’t measure how common it is. The 38-firm survey sometimes cited belongs to a separate products-and-services review. [FCA §5.1]

Scale assurance to impact, not firm size

  1. Consequence: What decision or action does AI influence?
  2. Consumers: Detriment, vulnerability, exclusion, market integrity.
  3. Autonomy: Recommends, interacts, or acts on its own?
  4. Data: Sensitivity, external processing, model dependencies.
  5. Reversibility: Can errors be spotted, contested and fixed?

Illustrative T3 risk tiers

  1. Tier A – Limited: Internal, reversible, low external impact. Output sampling, named owner, basic security and monitoring.
  2. Tier B – Material: Important workflows, confidential data, contested decisions. Structured evaluation, cross-functional approval, targeted scenarios.
  3. Tier C – High impact: Material consumer, market or resilience effects, or consequential autonomy. Independent challenge, stress tests, detailed evidence, enhanced monitoring.

Five lifecycle gates

  1. G0: Register and define purpose
  2. G1: Assess risk, design controls
  3. G2: Test and challenge
  4. G3: Authorise deployment
  5. G4: Monitor, change, retire

Smaller firms can use simpler tiering, representative samples and existing committees. Lean is credible when ownership, reasoning, checks and interventions are evidenced. [FCA §§2.1, 3.1, 3.3]

Twelve control domains

For each domain: what the FCA observed, the T3 standard, the evidence to keep and the test that proves it works. These are T3 recommendations, not new FCA rules.

C01: Use-case inventory and intended purpose

Know every AI system and agent you run, why it exists, and who owns it. (FCA §§2.1, 3.1, 3.5, 5 · Cases A, H)

What the FCA observed: Better outcomes where firms start from consumer need, target market and success measures. Weak: governance retrofitted after build, unclear ownership. Inventories should span in-house, embedded and third-party AI, and separate experiments from production.

T3 standard:

  1. Record ID, purpose, owner, affected parties, status, supplier, model(s) and permissions.
  2. Map the workflow — input → model → action → outcome — and mark where it abstains or escalates.
  3. State intended benefits and excluded uses. Ask whether a non-AI route would be safer.

Evidence to keep: Inventory export · dependency map · purpose and impact statement · tier rationale · approval and scope exclusions

Assurance test: Sample applications, embedded features and agent configurations. Trace each to the inventory, owner, real permissions and production use.

Scenario: 24 apps are approved. An expense-platform review finds an embedded summariser and two autonomous bots that aren’t registered. Reconcile procurement, identity permissions, contracts and usage to find the real control perimeter.

Proportionality: Small firms: one controlled register. Material use cases: full lineage and permission records.

Escalate if: Unregistered production AI · undocumented automated actions · no business owner

C02: Governance, accountability and SMF challenge

Prove end-to-end accountability and informed challenge. (FCA §§3.1, 3.3 · Cases H, I, J, L, M)

What the FCA observed: Good: early senior accountability, multidisciplinary governance, auditable reasoning, named business and technical owners. Poor: approval after deployment, generic committees with no AI-specific challenge, fragmented responsibility.

T3 standard:

  1. Name the accountable executive (SMF where applicable). Separate business, model, data and control owners.
  2. Define approval rights and independent challenge for launches and model changes. Keep the dissent and the actions.
  3. Give leaders concise MI on harm, performance, overrides, incidents and suppliers — not technical volume.

Evidence to keep: RACI and terms of reference · delegated authorities · papers and decisions · challenge logs · role training · accepted residual risks

Assurance test: Pick one high-risk decision. Rebuild its chronology and check whether a recorded concern actually changed the outcome.

Scenario: A high-impact customer tool is approved with accuracy findings still open. Who understood the residual risk? What alternatives were weighed? What conditions were set? A signature without the challenge history answers none of this.

Proportionality: Existing risk committees can do this if they show technical expertise, named accountability and authority to escalate.

Escalate if: No decision trail · nominal sign-off · issues closed without rationale

C03: Risk appetite, permissions and agent control

Turn risk appetite into hard limits and intervention triggers. (FCA §§3.1, 3.5 · Cases M, P)

What the FCA observed: Firms using differentiated thresholds, dashboards and incident playbooks — and registers of live agents showing each owner and the systems and data each can reach.

T3 standard:

  1. Set use-case metrics and tolerances: safety, consumer harm, service interruption, repeat errors.
  2. Fix autonomy limits: spend and action caps, approval gates, permitted tools, kill-switch rights.
  3. Agree amber and red actions before go-live — who suspends an agent, and how that is verified.

Evidence to keep: Appetite-to-metric mapping · thresholds · agent permissions list · runbook · escalation matrix · suspension and rollback test

Assurance test: Inject a simulated threshold breach and a disallowed agent action. Verify prevention, alerting, escalation and containment.

Scenario: An agent may draft a payment but not submit it. Can a fake “urgent” request, a prompt injection or a config change bypass that limit? Do operators see it? Can access be cut immediately?

Proportionality: Low-impact tools need fewer metrics. Every material autonomous action needs an explicit authority boundary.

Escalate if: Undefined thresholds · agent access beyond task need · no way to stop a harmful sequence

C04: Model evaluation and validation

Test performance against the intended outcome and foreseeable failures. (FCA §§2.1, 3.2 · Cases A, B, K)

What the FCA observed: Good: structured output criteria, human review alongside automated scoring, bias scenarios, narrow pilots before scaling. Weak: technical metrics detached from user outcomes.

T3 standard:

  1. Fix questions, success measures, thresholds and test data before testing starts.
  2. Cover normal, hard, outlier, counterfactual and abstention cases. Keep test sets independent.
  3. Log configurations, model and prompt versions, error taxonomy, reviewer judgement and retests.

Evidence to keep: Test plan · traceable dataset · pass/fail thresholds · versioned prompts and models · human-scored samples · exceptions and retests

Assurance test: Re-run a reproducible sample on the deployed version. Trace one failure cluster to its fix and regression result.

Scenario: A claims summariser scores 94% on routine cases but routinely drops a critical exclusion. A high average doesn’t justify deployment if critical omissions go undetected or the evaluation can’t be reproduced.

Proportionality: Curated representative samples can suffice for smaller firms. Scale coverage and independence with impact.

Escalate if: No acceptance threshold · no failure analysis · model accuracy measured, task outcome not

C05: Generative AI safety and adversarial testing

Find the failures specific to generative AI and agents. (FCA §§2.2, 3.2, 4.1 · Cases K, R)

What the FCA observed: Evaluation of completeness, faithfulness, reasoning and safety, plus pre-deployment attempts to make tools fail. No red-team taxonomy or penetration-test standard is prescribed.

T3 standard:

  1. Test unsupported answers, unsafe instructions, citation failures, data leakage and prompt-injection routes in your architecture.
  2. For tool-using agents: unauthorised calls, privilege boundaries, action chaining, unsafe recovery.
  3. Pair automated scoring with human review. With LLM-as-judge, check disagreement and correlation.

Evidence to keep: Attack model · adversarial prompt suite · recorded runs · severity rubric · guardrail configuration · adjudications and regressions

Assurance test: In a safe environment, attack through user input, retrieved content and tool responses. Verify logging, refusal and containment.

Scenario: An assistant reads external documents. A planted passage tries to extract internal data or call a tool. Capture the full request and tool trace, the control that acted, and an independent retest.

Proportionality: Choose attack classes from your threat model. A drafting tool and a payment agent need very different intensity.

Escalate if: Unauthorised action succeeds · sensitive data leaks · critical safeguard fails without remediation

C06: Fairness and differentiated outcomes

Find unjustified outcome differences — without guardrails creating new harm. (FCA §2.1 · Cases B, C)

What the FCA observed: Pre-deployment testing across consumer groups, vulnerability indicators and scenarios, followed by remediation and retest. In one case a guardrail cut drift but made recommendations too generic for some consumers.

T3 standard:

  1. Derive groups and scenarios from the real journey. Use protected or proxy variables only where lawful.
  2. Compare outcomes and errors. Check whether exclusions, escalations and false positives fall unevenly.
  3. Confirm fixes improve the whole journey — access and suitability — not one model score.

Evidence to keep: Fairness hypothesis and plan · lawful group-level results · vulnerability scenarios · decision rationale · retests and outcome monitoring

Assurance test: Replicate segmented results on a controlled sample. Root-cause one disparity and check the fix didn’t cause harm downstream.

Scenario: A fraud tool flags more customers with accessibility-related indicators. Are the indicators legitimate? Do false-positive costs differ by group? Does a revised threshold cut total harm without a new exclusion?

Proportionality: Where group measures aren’t meaningful or lawful, say why and use scenarios or qualitative review instead.

Escalate if: Unexplained material detriment · vulnerable users excluded without support · fixes that worsen outcomes

C07: Consumer understanding and vulnerable support

Make AI journeys understandable, accessible and contestable. (FCA §§2.2–2.3 · Cases D, E, F, G)

What the FCA observed: Good: point-of-use explanation of capability and limits, communications tested on real consumers, routes to human advisers, careful use of vulnerability markers. Poor: overstated reliability, consumers trapped in automated journeys.

T3 standard:

  1. Tell consumers when AI is involved, what it can’t do, and how to reach a person — in plain language.
  2. Test comprehension and friction with diverse users: disability, financial difficulty, low digital capability.
  3. Keep proportionate records of prompts, outputs, filters and escalations, within privacy and retention rules.

Evidence to keep: User notices · consumer-testing results · accessibility checks · approved sources · complaint and escalation logs · redress evidence

Assurance test: Run scripted scenarios, including failed comprehension and vulnerable needs. Check the human handoff keeps context.

Scenario: A customer in financial distress gets an accurate automated reply with no usable next step. Run the journey: are the limits clear, and is real human help within reach?

Proportionality: Internal wholesale workflows need a different control set. Don’t apply consumer obligations where they don’t legally apply.

Escalate if: Misleading capability claims · no viable human channel · adverse outcomes not understood

C08: Human oversight and automation bias

Show reviewers can spot, challenge and stop AI errors. (FCA §3.4 · Cases N, O)

What the FCA observed: Good: affordability reviewers judging cases before seeing the AI recommendation. Caution against tick-box review and gradual uncritical acceptance, especially where escalation criteria are vague.

T3 standard:

  1. Define which outputs need review, who is qualified, what they see and what authority they hold.
  2. Where suitable, use blinded human-first comparisons and seeded errors to measure real challenge.
  3. Log decisions and reasons. Read override rates in context — low overrides are not proof of success.

Evidence to keep: Reviewer instructions and competency · seeded-case results · blinded comparisons · override logs · escalations · refresher training

Assurance test: Give trained reviewers a set of plausible wrong outputs. Measure detection, justified override and escalation.

Scenario: Reviewing blind, complaint handlers catch 16 of 20 material errors. Shown the AI recommendation first, they catch 9. Investigate anchoring and training — “100% human approval” would be a misleading indicator.

Proportionality: Low-impact uses: spot checks. High-impact uses: capable reviewers with real power to intervene.

Escalate if: Injected error missed · reflexive sign-off · reviewer can’t stop the action

C09: Third-party AI and model supply chain

Keep accountability — with transparency, change visibility and an exit. (FCA §4.2 · Cases I, S, T)

What the FCA observed: Firms reducing single-model dependency and recording AI components. Criticised: unchallenged vendor assurances, and a retired foundation model forcing redesign with no transition ready.

T3 standard:

  1. Identify critical models, APIs, hosting, subprocessors and subcontractors for the real deployment.
  2. Contract for change notice, security, performance data, incident support and practical substitution rights.
  3. Test supplier claims against your use case. Rehearse one outage or substitution for each material service.

Evidence to keep: AI due diligence · contract terms · model and version register · change notices · post-update evaluations · exit tests

Assurance test: Trace one material supplier claim to evidence. Simulate a version change or outage and prove the fallback works.

Scenario: A provider swaps the foundation model. A generic assurance letter doesn’t show retrieval, thresholds, privacy controls or tone still hold. Run targeted regression tests and record a rollback decision.

Proportionality: Small firms can use lighter diligence, but must know their dependencies and the limits of vendor assurances.

Escalate if: Critical supplier claim untested · unannounced impactful update · no fallback for material use

C10: Operational and cyber resilience

Ensure AI degradation or loss doesn’t break important services. (FCA §4.1 · Cases Q, R · SYSC 15A)

What the FCA observed: Considered hosting, spare capacity, pre-deployment stress testing and AI dependencies mapped into resilience planning. SYSC 15A obligations apply to firms within scope.

T3 standard:

  1. Establish whether AI supports an important business service, and its role in disruption scenarios and impact tolerances.
  2. Test degraded output, outages, rate limits, vendor withdrawal and cyber compromise — not just total failure.
  3. Define graceful degradation, manual fallback, backlog handling, reconciliation and recovery owners.

Evidence to keep: Service dependency map · resilience scenarios · capacity and recovery tests · fallback runbook · cyber test results

Assurance test: Run a tabletop and, where safe, a technical failover. Measure outcome restoration, backlog and error spread — not uptime.

Scenario: During a provider slowdown a triage model stays up but returns incomplete priorities. Uptime looks healthy; outcomes don’t. Test degraded running, the fallback queue and recovery of mishandled cases.

Proportionality: Firm-specific rules decide what’s mandatory. Low-impact internal tools may need only a simple continuity route.

Escalate if: Unmapped material dependency · unrecoverable AI process · critical customers unsupported during failure

C11: Production monitoring and incident response

Catch deterioration early and act before harm spreads. (FCA §§2.3, 3.2–3.5 · Cases K, M, P)

What the FCA observed: Dashboards, spot checks, interaction logs and staff feedback. Criticised: purely technical, reactive dashboards, unjustified long gaps between reviews, vague escalation. No fixed review frequency is mandated.

T3 standard:

  1. Monitor consumer outcomes alongside technical performance: segmented errors, harmful outputs, overrides, complaints.
  2. Set baselines, thresholds, alert owners, severity levels, containment and criteria to retrain, roll back or suspend.
  3. Check monitoring catches new patterns. Test alerts and escalation end to end, periodically.

Evidence to keep: Outcome and risk dashboards · sampling records · thresholds · incidents, complaints and near misses · change decisions · audit logs

Assurance test: Inject a synthetic breach. Trace notification, triage, decision, action, closure and management reporting.

Scenario: Answer quality is falling, but alerts only fire on API downtime. Simulate a burst of harmful outputs: does the process intervene before complaints and losses build?

Proportionality: Set cadence by volatility, impact and detectability. Low-volume, high-severity uses need event-led escalation.

Escalate if: Red breach with no responder · deteriorating outcomes invisible · faulty tool runs unchallenged

C12: Change governance, review and retirement

Keep assurance valid as systems and processes change. (FCA §§3.1–3.2, 4.2 · Cases I, T)

What the FCA observed: Governance across scoping, design, testing, deployment, monitoring and decommissioning. Praised: specialist approval of material model changes and visibility of supplier updates. Shown: disruption when a dependent model retired.

T3 standard:

  1. Define change triggers: model swaps, material prompts, retrieval corpus, tool permissions, pipelines, hosting, purpose.
  2. Scale impact assessment, regression, approval and rollback to severity. Re-verify agent permissions after updates.
  3. Plan retirement: revoke access, handle data, inform customers where relevant, archive evidence.

Evidence to keep: Change register · model, prompt and version lineage · impact assessments · regression packs · sign-offs · rollback tests · exit records

Assurance test: Take a recent supplier or prompt change. Prove the impact assessment came before release and tested material failure modes.

Scenario: A supplier retires the model behind a claims workflow. Was notice received? Was the dependency inventory accurate? Was an alternative tested and validated before cutover?

Proportionality: Low-impact changes: light approval. Critical permissions and consumer-impacting updates: structured challenge.

Escalate if: Uncontrolled production update · no rollback · retired model breaks an important workflow

Measuring effectiveness

Evidence, tests and ratings

A policy proves intent. Only tested outcomes prove effectiveness.

  1. E0 – Not evidenced: No current, relevant evidence. No conclusion possible.
  2. E1 – Designed: Objective, owner and procedure exist. Operation unproven.
  3. E2 – Operating: Records show it runs. Effectiveness or coverage not shown.
  4. E3 – Effective: Repeatable tests and outcomes show risk is mitigated.
  5. E4 – Enhanced: Plus independent challenge, trends, change resilience, measured improvement.

This is an advisory scale, not an FCA scale. Rate design, operation and effectiveness separately. Never average away a material red issue. Justify every “not applicable”.

Minimum evidence pack for a material use case

  1. Scope: Purpose, tier, population, named owners
  2. Lineage: Architecture, data, models, providers, agent permissions
  3. Risk: Impact assessment, control design, thresholds, approval
  4. Validation: Plan, datasets, results, failures, fixes, regression
  5. People: Human oversight, consumer outcomes, vendor diligence
  6. Live: Monitoring, incidents, changes, resilience, retirement

Sampling integrity: For every test record the period, population, sample method, tester, exceptions, remediation owner, due date and retest. A policy alone never earns “effective”.

Worked example: AI-drafted complaint replies

An LLM drafts replies from approved policy and case records. It never sends. A trained handler approves every letter. Assess Consumer Duty and DISP in the firm’s own context; treat data protection and retention separately.

Five tests:

  1. Correctness: Check drafts against case facts, policy and redress decision. Classify misstatements and invented reasons.
  2. Understanding: Test letters with representative consumers, including accessibility and low financial capability.
  3. Oversight: Seed plausible errors. Measure detection, correction and escalation against human-first judgement.
  4. Safety and privacy: Test confidential-data handling, instructions hidden in source documents, and improper disclosure.
  5. Live controls: Verify draft traceability, complaint-trend monitoring, escalation thresholds and change reviews.

Acceptance logic (illustrative, not FCA thresholds):

  1. No automatic sending.
  2. Any misleading outcome or unauthorised disclosure is an escalation event.
  3. Every high-severity defect fixed and retested before release.

The firm sets — and justifies — its own tolerance for lower-severity language issues.

A credible conclusion states what was tested, over what period, with which limits — which controls passed, which failures remain, and who accepted the residual risk. Not simply that the tool is “compliant”.

From baseline to routine: a 90-day plan

Baseline. Prove two controls in live use. Scale only where evidence supports it.

  1. Baseline (days 1–15): Validate the inventory, classify a first sample, map regulated activities, consumer touchpoints and critical dependencies. Name owners.
  2. Prioritise (days 16–30): Pick two material use cases. Document benefits, harms and legal applicability. Agree thresholds and evidence owners.
  3. Design (days 31–45): Build test suites: task quality, harmful errors, human challenge, vendor changes, consumer outcomes.
  4. Prove (days 46–60): Run tests, collect outcome evidence, triage and retest fixes. Dry-run escalation and fallback.
  5. Decide (days 61–75): Present conclusions, open limitations and remediation options to the accountable governance body.
  6. Embed (days 76–90): Publish a minimum control library, monitoring owners, review triggers and a change-and-retirement standard.

By day 90, show one complete trace: approved purpose → real configuration → reliable testing → informed go-live → outcome monitoring → tested intervention.

One-page board / SMF pack

  1. Adoption: Use cases, tiers, live agents, dependencies
  2. Outcomes: Service and consumer impact, good and bad
  3. Exceptions: Open high risks, overdue fixes, accepted risks
  4. Control tests: Coverage, results, challenge outcomes
  5. Resilience: Model changes, outages, failover results
  6. Decisions: Approve, restrict, fix, defer, suspend, retire

FCA crosswalk and sources

Map each control domain to the FCA text. Check firm-specific binding rules separately.

  1. FCA §2.1 (Cases A–C): C01 Inventory · C04 Evaluation · C06 Fairness
  2. FCA §§2.2–2.3 (Cases D–G): C07 Consumer understanding and support
  3. FCA §3.1 (Cases H–J): C01 Inventory · C02 Governance · C03 Agents · C12 Change
  4. FCA §3.2 (Case K): C04 Evaluation · C05 GenAI testing · C11 Monitoring
  5. FCA §3.3 (Cases L–M): C02 SMF accountability · C03 Thresholds
  6. FCA §3.4 (Cases N–O): C08 Human oversight
  7. FCA §3.5 (Case P): C03 Risk appetite · C11 Monitoring
  8. FCA §4.1 (Cases Q–R): C05 Adversarial testing · C10 Resilience
  9. FCA §4.2 (Cases S–T): C09 Third-party AI · C12 Change and exit

Primary sources (checked 21 October 2026)

  1. FCA, AI in financial services: good practice and areas for improvement, 19 Oct 2026, updated 20 Oct 2026.
  2. FCA, AI and the FCA: our approach, updated 2 Oct 2026.
  3. FCA, AI in financial services: shaping our approach through industry engagement, 8 Jun 2026.
  4. FCA Handbook, SYSC 15A (firms in scope).
  5. FCA, The Mills Review, July 2026.

Use and limitations

This guide operationalises selected themes, not every requirement that applies to a given firm. FCA case studies are anonymised observations. T3 test protocols, risk tiers and E0–E4 ratings are proposed practices only.

Independent guidance by T3 Consultants for boards, SMFs, CROs, CISOs, risk, compliance and internal audit. Not issued, approved or endorsed by the FCA, and not a substitute for legal or regulatory advice.