Build & Operate AI · Layer 05
AI agent governance: bounded, tested and traceable
An agent is a loop with your permissions. It reads, plans, calls tools and acts, then does it again, often faster than anyone can review. Controls written for chatbots do not reach inside that loop.
We help fintechs, technology companies and asset managers put agents into production safely: every agent registered, given only the access its task needs, bounded by guardrails at the tool boundary, attacked before launch, and leaving a trace you can replay.
01 · What good looks like
What is good AI agent governance?
AI agent governance is bounding, testing and evidencing what an autonomous AI system is allowed to do, at the point in its loop where each action is actually taken.
It is where AI governance stops being a documentation exercise and becomes an engineering one. Good agent governance has five parts:
- A register of every agent, the tools and MCP servers it can reach, and how far a wrong action could travel
- Its own scoped identity and least agency: only the tools and permissions the task needs
- Deterministic guardrails at the tool boundary that can allow, gate or deny an action before it lands
- Human oversight matched to consequence: in the loop for irreversible actions, on the loop for the rest
- Trace-level logging, evaluations and a kill switch that actually reaches the runtime
You can tell it is working when three things are true:
- Least agency. No agent can reach a tool, system or permission its task does not need.
- Limits in code, not in the prompt. A persuaded model still cannot take an action the guardrail denies.
- Every run can be replayed. You can show exactly what an agent did, with which data, and why it was allowed.
Every action an agent attempts then meets one of four outcomes:
02 · Why do it
Why an agent is not a chatbot with extra steps
Every control built for generative AI assumed a person read the output before anything happened. Agents remove that person. Four failure modes follow:
- Borrowed identity, full authority. An agent running on a user's credentials can reach everything that user can, at machine speed.
- Instructions hidden in what it reads. Pages, tickets and tool responses can carry commands, and memory lets one poisoned entry persist.
- No trace, no reconstruction. A chat transcript does not show tool calls, arguments or records touched.
- Oversight that becomes a click. An approval confirmed fifty times a day stops being oversight by the tenth.
Source: Gartner, June 2025.
The state of the art: what standards and regulators now expect of agents
Open industry guidance, December 2025
Ten agent-specific risks, led by agent goal hijack, tool misuse and identity and privilege abuse, through to memory poisoning, cascading failures and rogue agents.
The checklist your red team, engineers and auditors will share.
IMDA Model AI Governance Framework for Agentic AI, January 2026
The first government framework written for agents: scope risks early, keep clear human responsibility, build technical safeguards across the lifecycle, and inform end users.
A practical template other regulators are likely to follow.
Firms using or providing AI in the EU
Human oversight (Article 14) and automatic logging (Article 12) apply to high-risk systems, including agents used for decisions such as credit.
Oversight and traceability have to work at agent speed.
US banks, April 2026 model risk guidance
The new guidance excludes generative and agentic AI and leaves them to broader risk management, with a request for information on AI to follow.
Banks need an agent framework of their own, now.
UK firms
No AI-specific rules. Consumer Duty and SM&CR apply, and the FCA is reviewing how senior managers oversee AI.
A named senior manager must be able to explain what agents are allowed to do.
AI management standard and US risk framework
Both expect risk assessment, defined human oversight and monitoring, which agents make harder to evidence.
Agent controls produce the evidence certification needs.
Summaries for orientation, as of October 2026. Check how each applies to your firm. Primary sources: OWASP Top 10 for Agentic Applications, IMDA Model AI Governance Framework for Agentic AI, EU AI Act (Regulation (EU) 2024/1689), Federal Reserve SR 26-2, ISO/IEC 42001, NIST AI RMF.
Reviewed by Lana Hampicke, Co-founder and Managing Partner, AI Safety & Governance, T3. Last updated October 2026.
03 · How to decide
Seven decisions before an agent goes live
Agent governance is mostly a set of design decisions. Make them before launch and the controls are cheap. Make them after an incident and they are not.
Decision 1 · InventoryWhich agents exist, and what can they reach?
List every agent, including ones built in SaaS tools and by individual teams, with the tools, connectors and MCP servers each can call.
Decision 2 · Autonomy levelHow autonomous should each agent be?
Choose per use case: suggest only, act with approval, act and report, or act alone. Higher autonomy needs stronger controls.
Decision 3 · IdentityWhose identity does it act under?
Give each agent its own scoped identity and credentials rather than borrowing a user’s full access.
Decision 4 · Least agencyWhich tools and actions may it take?
Set allow-lists, spend and rate limits, and deny rules in code at the tool boundary.
Decision 5 · OversightWhich actions need a human?
Put people in the loop for irreversible or high-impact actions, and on the loop for the rest, so approval stays meaningful.
Decision 6 · ContainmentHow do we stop it?
Design circuit breakers, blast-radius limits and a kill switch that reaches the running agent, and test them.
Decision 7 · EvidenceHow do we prove what it did?
Log every step of every run, run evaluations when models or tools change, and keep traces long enough to replay an incident.
What already exists
Tools you may already have
The agent tooling market is moving fast. Most of these are worth having, but each covers one stage of the loop. The gaps between them are where agent incidents start.
Agent platform guardrails
Gives youBuilt-in policies, content filters and approval steps from your agent platform.
MissesYour business rules, cross-platform agents, and attacks designed to get past the defaults.
AI and MCP gateways
Gives youA central point to inspect, log and restrict calls between agents, models and tools.
MissesWhether each agent should have that access in the first place.
Identity and access management
Gives youStrong identity, secrets and privileged access controls you already run.
MissesAgent-specific scoping and delegation chains, which most IAM set-ups were not designed for.
Observability and tracing tools
Gives youDetailed traces of agent runs, prompts and tool calls for debugging and evaluation.
MissesWhat should have been prevented. Tracing records the incident; it does not stop it.
Automated agent red-teaming
Gives youFast, repeatable tests against known agent attacks.
MissesChained, business-specific attacks that need a human to design.
Agent policies on paper
Gives youA statement of intent on how agents should be used.
MissesAnything enforced at runtime. A policy does not intercept a tool call.
Categories are illustrative. T3 holds no commercial ties to the agent platforms we test.
Putting the decisions together
What this usually means for your firm
The right controls depend on what your agents can touch and who you answer to.
Fintech
Agents in onboarding, payments, support and fraud operations.
- Start with
- Agent register and blast-radius scoring, then approval gating on anything that moves money or changes a customer record.
- Watch for
- Agents acting on customer accounts with staff credentials, and injection through customer documents and emails.
- Regulatory lens
- FCA Consumer Duty and SM&CR, DORA for EU entities, and EU AI Act oversight for credit decisions.
Technology company
Agents in products, coding and internal operations, many MCP servers.
- Start with
- Non-human identities, tool allow-lists and MCP server review, with evaluations in your release pipeline.
- Watch for
- Coding agents with production access, unreviewed MCP servers, and model updates that change agent behaviour.
- Regulatory lens
- OWASP Agentic Top 10, ISO/IEC 42001, and EU AI Act provider duties.
Asset manager
Copilot agents, research agents and operations automation.
- Start with
- Scoping what Copilot and research agents can read and do, with trace logging for anything client-facing.
- Watch for
- Agents reading untrusted research content, and agents with access to client data or order systems.
- Regulatory lens
- SEC policies and supervision for US advisers, the FCA, and DORA for EU firms.
04 · How we help
How we help: bound, attack, evidence
We find where in the loop your controls actually attach, test whether they hold, and build the ones that are missing, in your execution path rather than in a policy document.
We hold no commercial ties to the agent platforms we test, and our testers contribute to OWASP GenAI red-teaming work.
Agent discovery and blast-radius mapping
T3 with your platform owners
Shows where to focus first
Identity, guardrail and trace design
T3, built to be implemented by your engineers
Controls in code, not policy
Adversarial testing of the loop
T3
Needs attacker skill and agent depth together
Evaluations and continuous assurance
Your engineers, reviewed by T3
Keeps controls current as agents change
Depth follows blast radius. An agent that can move money gets gating, a full red team and replayable traces. An agent that drafts internal summaries gets scoping and a targeted test.
What you get
Five deliverables
Each one is usable on its own. Together they let you put agents into production and show exactly why they were allowed to act.
- 01
Agent Register
Every live and planned agent, its owner, the tools, connectors and MCP servers it can reach, and a blast-radius score.
OWASP ASI03Singapore agentic framework - 02
Agent Permission & Guardrail Spec
Scoped identities, tool allow-lists, deny rules, approval thresholds and spend limits, written to be implemented by your engineers.
OWASP ASI02 and ASI03EU AI Act Art. 14 - 03
MCP Server Security Review
Review of the MCP servers and connectors your agents use: permissions, provenance, and exposure to tool poisoning.
OWASP ASI04DORA third-party risk - 04
Agent Red Team Report
Goal hijack, tool misuse, memory poisoning and privilege escalation tested across the loop, with reproduction steps and a re-test.
OWASP Agentic Top 10MITRE ATLAS - 05
Kill Switch Design
Circuit breakers, blast-radius limits and a tested way to halt any agent at runtime, with an incident playbook.
OWASP ASI08 and ASI10EU AI Act Art. 14
Timing
From discovery to tested controls in about ten weeks
Agent discovery, tool and MCP mapping, and blast-radius scoring, including agents adopted without approval.
Identity model, guardrail specification, oversight design and trace logging standard.
Adversarial testing across the loop, remediation support and re-test.
After that, regression evaluations and quarterly reviews keep controls current as agents gain capability. Agent assurance is not a point-in-time exercise.
05 · Questions
What leaders ask us
How is agent governance different from AI model testing?
Model testing asks whether an output is accurate, fair and safe. Agent governance asks what the system is allowed to do with that output, whether those permissions hold under attack, and whether every action can be reconstructed afterwards.
We use a vendor’s agent platform. Is that not their responsibility?
The platform provides capability. The permissions, approval thresholds and audit trail sit with you, and so does the regulatory obligation. Vendor assurance covers their controls, not your configuration of them.
Can we not just tell the agent to behave in its prompt?
No. A prompt is a request to a probabilistic system, and anything that can be persuaded can be persuaded by an attacker. Limits belong in code at the tool boundary, with the prompt as a second layer.
We use MCP servers. Does that change the risk?
Yes. Every connected MCP server is another set of tools the agent can reach, often with permissions inherited from whoever connected it. Our register and MCP security review cover them explicitly.
Do you test agents in production?
Normally in a non-production environment with equivalent permissions. Where production testing is genuinely needed, it is scoped, time-boxed and agreed with your risk function in advance.
What does the EU AI Act require for agents?
It depends on the use case. Where an agent is part of a high-risk system, such as credit decisions, human oversight and logging requirements apply. We assess classification at the start rather than assuming it.
What is the OWASP Top 10 for Agentic Applications?
Open industry guidance published by the OWASP GenAI Security Project in December 2025. It lists ten agent-specific risks, led by agent goal hijack, tool misuse and identity and privilege abuse, through to memory poisoning, cascading failures and rogue agents. We use it as the shared checklist for red teaming, engineering and audit.
How long does an AI agent governance engagement take?
About ten weeks from discovery to tested controls: agent discovery and blast-radius mapping in weeks 1 to 2, identity, guardrail and trace design in weeks 3 to 6, and adversarial testing, remediation and re-test in weeks 7 to 10. Regression evaluations and quarterly reviews follow.
Who needs AI agent governance?
Any firm running agents that can read sensitive data, call tools or change real state, such as payments, customer records, order systems or code. We work with fintechs, technology companies and asset managers in regulated markets.
Get started
Ready to prove your agents are under control?
A 45-minute call with a T3 agent specialist. We will map where your agents are running, what they can reach and under whose permissions, and outline a realistic first step.
