Govern AI · Layer 03
AI security: configured, defended and attacked first
AI systems take instructions from the data they read, hold keys to the systems they connect to, and depend on models, plugins and MCP servers you did not write. Traditional application security was not built for any of that.
We secure LLM applications and AI agents from both sides: hardening configuration, access and guardrails, then attacking them the way a real adversary would, aligned to the OWASP Top 10 for LLM and Agentic Applications and MITRE ATLAS.
01 · What good looks like
What is good AI security?
AI security is protecting AI systems, and the data and systems they can reach, from being manipulated, leaking information or taking actions they should not.
It has two halves. Defensive security makes the system hard to misuse. Offensive security proves it. Good AI security covers five areas:
- Secure configuration of every LLM, platform and AI feature, reviewed against a baseline
- A threat model for each AI system, covering data, prompts, tools and agents
- Identity and access designed for AI: scoped credentials, least privilege, secrets out of code
- Guardrails on what goes in and what comes out, enforced in code rather than in the prompt
- Adversarial testing that proves the controls hold, then re-tests after every material change
You can tell an AI security programme is working when three things are true:
- Secure by default. New AI systems start from a hardened baseline, not from the vendor's defaults.
- Least agency. Every model and agent can reach only what its task needs, with its own scoped identity.
- Attacked before attackers do. Prompt injection, jailbreaks and tool abuse are tested before launch and after every change.
Every finding then gets one of four responses:
02 · Why do it
Why AI needs its own security
Firewalls, scanners and penetration tests were built for software that does what its code says. AI adds three new attack surfaces:
- Instructions arrive through data. A web page, email or document the AI reads can carry hidden instructions. This is indirect prompt injection.
- AI holds the keys. Agents call tools and APIs with real permissions. Too much access turns a manipulated prompt into a real-world action.
- The supply chain is new. Third-party models, plugins and MCP servers run inside your environment, often with little review.
Source: IBM Cost of a Data Breach Report 2025.
The state of the art: what attackers, standards and regulators now expect
Open industry guidance, 2026 editions
The Top 10 for LLM Applications (2026 edition) ranks prompt injection first and excessive agency third. A separate Top 10 for Agentic Applications, released December 2025, covers goal hijacking, tool misuse and privilege abuse.
The common language your engineers, testers and auditors will use.
Open knowledge base of real attacks on AI
Maps adversary tactics and techniques against AI systems, in the same style as MITRE ATT&CK.
Lets you plan red teaming around how attackers actually operate.
US government taxonomy, March 2025 edition
Defines adversarial machine learning attacks, including evasion, poisoning, privacy attacks and attacks on generative AI, with mitigations.
A precise vocabulary for threat models and findings.
UK AI Cyber Security Code of Practice, now ETSI TS 104 223
Thirteen baseline security principles across secure design, development, deployment, maintenance and end of life.
The emerging baseline UK and European buyers will ask suppliers about.
Providers of high-risk AI used in the EU
Article 15 requires accuracy, robustness and cybersecurity, including resilience against data poisoning, adversarial inputs and model flaws.
Security testing becomes part of conformity for high-risk systems.
EU financial firms, including asset managers and payment fintechs
Requires ICT risk management and digital operational resilience testing, with threat-led penetration testing for the most significant firms.
AI systems that support important functions fall inside your testing programme.
Certifiable security and AI management standards
ISO/IEC 27001 covers information security controls. ISO/IEC 42001 adds AI-specific risk and impact assessment.
AI security evidence supports both certifications.
Summaries for orientation, as of October 2026. T3 contributes to OWASP and MITRE work on AI threat taxonomies.
03 · Defend and attack
The two halves of AI security
Defensive work makes AI hard to misuse. Offensive work proves whether it is. One without the other leaves you guessing.
Defensive security
Configure, control, monitor
LLM and platform configuration
System prompts, model settings, retrieval sources, logging and data retention set to a hardened baseline.
Identity, access and secrets
Each model and agent gets its own scoped identity. API keys move to a secret store with rotation and spend limits.
Guardrails and output handling
Input filtering, output validation and action gating enforced in code, so a persuaded model cannot act alone.
Agents, tools and MCP servers
Tool allow-lists, least agency, and confirmation steps for irreversible actions.
Vendor and supply chain
Security review of model providers, plugins and AI features inside SaaS before they touch your data.
Monitoring and response
Prompt and tool-call logging, abuse detection, and an incident playbook that covers AI.
Offensive security
Threat model, attack, re-test
Threat modelling
Map how each system could be attacked, using OWASP, MITRE ATLAS and NIST AI 100-2.
Prompt injection testing
Direct and indirect injection through documents, web pages, emails and retrieved data.
Jailbreak and policy bypass
Attempts to make the system ignore its rules, reveal its system prompt or produce harmful output.
Data extraction
Attempts to pull sensitive data, other users’ content or training data out of the system.
Agent and tool abuse
Goal hijacking, tool misuse and privilege escalation across agents, tools and MCP servers.
Re-test and regression
Every fix re-tested, with an attack suite you can re-run when models or prompts change.
What already exists
Tools you may already have
The AI security market is moving fast and you may already own several of these. Each one covers part of the attack surface. None of them replaces a threat model and a human-led test.
Cloud provider guardrails
Gives youBuilt-in content filters, prompt shields and safety settings from your model provider.
MissesYour application logic, your data, and attacks designed to get past the default filters.
AI security gateways and firewalls
Gives youRuntime inspection of prompts and responses, with blocking and logging.
MissesOver-privileged agents and design flaws. A gateway cannot fix what the system is allowed to do.
Automated AI red-teaming tools
Gives youLarge libraries of known attacks, run quickly and repeatably.
MissesBusiness context and chained attacks that need a human to design.
Your existing penetration testing
Gives youStrong coverage of the web application, APIs and infrastructure around the AI.
MissesPrompt injection, jailbreaks and agent abuse, which most test scopes leave out.
Security posture tools
Gives youDiscovery of AI services and misconfigurations across cloud and SaaS.
MissesWhether the configuration actually holds under attack.
Vendor attestations
Gives youSOC 2 reports, ISO certificates and security questionnaires from AI suppliers.
MissesHow the vendor’s AI behaves inside your environment, with your data.
Categories are illustrative. T3 is independent of AI security vendors.
Where to start
What this usually means for your firm
The priorities depend on what your AI can reach and who you answer to.
Fintech
Customer-facing AI, fast releases, cloud-native stack.
- Start with
- Prompt injection testing of customer chatbots and onboarding flows, plus a configuration review of your model provider settings.
- Watch for
- Indirect injection through customer documents, data leakage between customers, and fraud using AI-generated content.
- Regulatory lens
- DORA testing for EU entities, FCA operational resilience, and EU AI Act Article 15 if you score credit.
Technology company
AI built into products, internal agents and MCP servers.
- Start with
- Threat models for each AI feature, agent and MCP security review, and an attack suite in your release pipeline.
- Watch for
- Agents with broad permissions, unreviewed MCP servers, and model updates that re-open fixed vulnerabilities.
- Regulatory lens
- OWASP LLM and Agentic Top 10, ETSI TS 104 223, ISO/IEC 27001 and 42001.
Asset manager
Microsoft 365 and Copilot, research tools, many data vendors.
- Start with
- Copilot and SaaS AI configuration review, data access scoping, and vendor AI security reviews.
- Watch for
- Copilot surfacing client data to the wrong people, and research tools reading untrusted content.
- Regulatory lens
- DORA, SEC information security priorities for US advisers, and FCA operational resilience.
04 · How we help
How we help: harden, then attack
We start with configuration and access, because that is where most AI exposures are cheapest to fix. Then we threat model each system and attack it the way a real adversary would.
Our testers contribute to the OWASP GenAI Red Teaming work and MITRE threat taxonomies, so testing follows current attacker practice rather than last year’s checklist.
Secure configuration review
T3 with your engineers
Fixes the cheapest problems first
Threat modelling
T3, using OWASP, MITRE ATLAS and NIST AI 100-2
Focuses testing where it matters
Adversarial testing and red teaming
T3
Needs attacker skill and AI depth together
Remediation and re-test
Your engineers, re-tested by T3
Proves the fix holds
Depth follows exposure. A customer-facing agent that can move money gets a full red team. An internal summariser gets a configuration review and a targeted test.
- 1Direct prompt injection, jailbreaks
- 2Indirect injection, poisoned data
- 3Model theft, supply chain
- 4Tool misuse, excessive agency
- 5Data leakage, unsafe output
What you get
Five deliverables
Each one is usable on its own. Together they give you a secured AI estate and the evidence to show it.
- 01
LLM Configuration Review
Model, platform and AI feature settings checked against a hardened baseline, including Copilot and SaaS AI, with prioritised fixes.
ETSI TS 104 223ISO/IEC 27001 - 02
AI Threat Model
How each AI system could be attacked, from data and prompts to tools and agents, with the controls that close each path.
MITRE ATLASNIST AI 100-2 - 03
AI Access Control Design
Scoped identities for models and agents, least-agency tool permissions, and secrets moved out of code.
OWASP Agentic Top 10DORA - 04
Prompt Injection Testing
Direct and indirect injection, jailbreaks and data extraction, with reproduction steps, severity ratings and a re-test.
OWASP LLM Top 10EU AI Act Art. 15 - 05
AI Vendor Security Review
Security review of model providers, plugins, MCP servers and AI features in SaaS before they reach your data.
ISO/IEC 42001DORA third-party risk
Timing
From review to re-test in about six weeks
Configuration and access review across your AI systems, plus threat models for the highest-exposure ones.
Adversarial testing: prompt injection, jailbreaks, data extraction and agent abuse.
Findings and remediation support, then a re-test of every fix.
After that, we leave you a re-runnable attack suite and can re-test when models, prompts or tools change.
05 · Questions
What leaders ask us
What is prompt injection?
Prompt injection is when someone gets an AI system to follow their instructions instead of yours. It can come directly from a user, or indirectly from a document, web page or email the AI reads. It is ranked first in the OWASP Top 10 for LLM Applications.
Is AI red teaming different from a penetration test?
Yes. A penetration test covers the application and infrastructure. AI red teaming attacks the model’s behaviour: prompt injection, jailbreaks, data extraction and agent abuse. Most firms need both, and we can run them together with your existing testers.
Our model provider has built-in guardrails. Is that enough?
Provider guardrails are a useful first layer, but they do not know your data, your tools or your business rules, and they are designed to be general. Attacks are tuned to get past them, so your own controls and testing still matter.
How do you secure AI agents and MCP servers?
By giving each agent its own scoped identity, limiting which tools and MCP servers it can reach, gating irreversible actions, and testing for goal hijacking and tool misuse as described in the OWASP Top 10 for Agentic Applications.
How often should AI systems be re-tested?
At least yearly for high-exposure systems, and whenever a model, system prompt, tool or data source changes materially. A re-runnable attack suite makes that cheap.
Do you test Microsoft Copilot and other SaaS AI?
Yes. We review configuration and data access for Copilot and AI features in your SaaS tools, and test what an attacker could make them reveal or do with your data.
Get started
Talk to us about securing your AI
A 45-minute call with a T3 AI security specialist. We will look at what your AI can reach, where it is most exposed, and what a sensible first test would be.
