Testing & evaluation
Does it work?
- Behavioural tests against golden datasets and versioned expectations
- Hallucination, grounding and retrieval-accuracy scoring
- Regression gates in CI/CD before any model or prompt change ships
Guide · Foundations
What is AI assurance?
AI assurance is the discipline of proving — with technical evidence rather than assertion — that an AI system does what it is supposed to do, refuses what it must not do, and stays that way in production. This guide covers the definition, the five pillars of an AI assurance framework, and how the lifecycle works in regulated environments.
- Reading time
- 8 minutes
- Audience
- Risk, engineering, compliance
- Standards
- EU AI Act · NIST AI RMF · ISO 42001
- Level
- Foundational
Definition
Assurance is evidence, not confidence.
An AI system is assured when a third party can inspect the record and reach the same conclusion the builder did. That record has to be technical: the evaluations that ran, the adversarial attacks that failed, the traces of what the agent actually did in production, and the version of the model and prompt that produced each result.
Traditional software assurance could rely on determinism — the same input produced the same output, so a passing test stayed passing. Generative and agentic systems are probabilistic, non-stationary and exposed to adversarial input at inference time. A single pre-launch sign-off says almost nothing about behaviour a month later.
That is why AI assurance is defined as a continuous practice rather than a gate. It runs before deployment, during operation, and again on every material change.
Why it matters
In high-stakes environments, an unproven AI system is an unpriced liability.
Healthcare
A summarisation error or an ungrounded recommendation is a clinical safety event. Evidence of grounding accuracy and human oversight is the difference between an adoptable tool and an unusable one.
Financial services
Supervisors expect explainability, fairness testing and auditable decision lineage. Assurance turns those expectations into artefacts that already exist when the examiner asks.
Energy, defence and public sector
Autonomy touches physical systems and sovereign data. Supervision levels, certified runbooks and immutable logs are prerequisites, not enhancements.
The framework
Five pillars of an AI assurance framework.
Most published frameworks — the EU AI Act's technical documentation duties, NIST AI RMF's Govern/Map/Measure/Manage functions, ISO/IEC 42001's management-system controls — decompose into the same five practical capabilities.
Security & red teaming
Can it be broken?
- Prompt injection, jailbreaks and multi-turn manipulation
- RAG poisoning, tool-chain injection and context leakage
- Continuous adversarial testing rather than a point-in-time pen test
Compliance & risk
Is it lawful?
- Crosswalks to the EU AI Act, NIST AI RMF and ISO/IEC 42001
- Sector overlays such as HIPAA, GDPR and financial-conduct rules
- Risk scoring tied to the technical evidence that produced it
Observability
What is it doing now?
- Traces of prompts, tool calls, retrievals and agent decisions
- Drift, cost and behaviour monitored against a certified baseline
- Incident reconstruction with full lineage, not sampled logs
Governance
Who is accountable?
- Model and agent inventory with owners, purpose and risk class
- Approval workflows and human-oversight checkpoints
- Evidence packs exported for auditors and regulators on demand
The lifecycle
How AI assurance works in practice.
Define intent
State what the system must do, what it must never do, and which regulations apply. Without a written intent there is nothing to assure against.
Step 01
Test before deployment
Evaluate accuracy, grounding, bias and safety against representative datasets, then red-team the system adversarially.
Step 02
Certify and record
Capture the evidence — test runs, findings, sign-offs — as immutable lineage attached to a specific model and configuration version.
Step 03
Monitor in production
Watch for drift, novel attacks and behavioural change against the certified baseline. Assurance that stops at launch is not assurance.
Step 04
Re-certify on change
Any model swap, prompt edit or new tool reopens the loop. Evidence accumulates as institutional memory rather than expiring.
Step 05
Common mistakes
Where assurance programmes fail.
Questionnaire compliancePolicy documents describe intent; they do not prove behaviour. Regulators increasingly ask for the technical artefact behind the answer.
One-off red teamingAn annual pen test cannot cover a model that is updated weekly and attacked continuously.
Disconnected toolingWhen evaluation, security and compliance data live in separate systems, no one can trace a finding to the control it breaks.
Assurance stopping at launchDrift, prompt changes and new tools silently invalidate the certification the system launched with.
Questions
Frequently asked.
- What is AI assurance?
- AI assurance is the practice of generating verifiable, technical evidence that an AI system behaves as intended — safely, accurately and within regulation — across its entire lifecycle, from pre-deployment testing through continuous production monitoring and re-certification.
- How is AI assurance different from AI governance?
- Governance defines policy: who owns which system, what risk is acceptable, which controls apply. Assurance is the evidence layer that proves those policies hold in running systems. Governance sets the rule; assurance produces the proof.
- What is an AI assurance framework?
- A structured set of controls, tests and evidence requirements spanning testing, security, compliance, observability and governance — mapped to standards such as the EU AI Act, NIST AI RMF and ISO/IEC 42001, so each control maps to the artefact that satisfies it.
- Is AI assurance the same as model evaluation?
- No. Evaluation measures model quality on benchmarks. Assurance covers the whole deployed system — retrieval, tools, agents, guardrails, human oversight — and carries the evidence forward over time.
- When should an enterprise start?
- Before the first high-stakes deployment. Retrofitting evidence onto agents already in production is significantly more expensive than instrumenting them from the start.
Next
See what an assurance platform looks like.
Next steps
Bring your hardest AI system. We will assure it.
Technical deep-dive on your live agents, an 8–12 week pilot with agreed success metrics, and partnership or investment conversations.