Before anyone signs off on an AI agent, they should see the evidence.
We give CIOs, compliance teams and CX leaders a go-live gate for every LLM agent: the risks written down, the tests that cover them, the results, and a record of who approved them. After launch, every real conversation is scored, and every change is re-tested before it reaches a customer.
The problem
Agents reach production on the strength of a good demo
A demo shows what an agent does when things go well. It does not show what happens when a customer pushes for a price, switches language, or asks for something the agent should refuse. When the CIO or the compliance lead is asked to approve, they usually have a recording and a vendor's assurance, nothing they could show an auditor.
- No written list of what the agent must never do.
- No acceptance criteria agreed with the business before launch.
- Prompt and model changes shipped without being re-tested.
- Problems found through customer complaints, not monitoring.
What we deliver
Six controls between an AI agent and your customers
A risk register per agent.
What could go wrong, in your terms: invented prices, unauthorised promises, data leaving through a tool, the wrong language or tone. Each risk has an owner and a test.
Acceptance criteria agreed up front.
Pass/fail criteria written with the business, CX and compliance before the agent is built, so "ready" means the same thing to everyone.
Pre-go-live testing on a copy.
Scripted customers play every risk against a disposable copy of the agent. Production is never touched, and every transcript is kept.
A sign-off pack.
Results against each criterion, the failures and how they were fixed, the configuration that was tested, and who approved it. You get one document for the go-live decision and the audit file.
Monitoring after go-live.
Every real conversation is scored on resolution, accuracy, handover, tone and efficiency, with reasons. Flags fill a review queue, and fixes count only when the problem stops recurring.
Change control and audit trails.
Each prompt or model change is re-tested before release, configurations are snapshotted and reversible, and rubric versions are stored with every score. Tool calls and decisions can be retrieved for an auditor.
How it works
Map the risk. Test it. Sign it off. Watch it.
- 01
Map the risk.
We work with the agent's business owner, CX and compliance to list what it must and must never do, and turn each item into a testable criterion.
- 02
Test before go-live.
The suite runs against a copy of the agent. Failures are fixed and re-run until every critical criterion passes.
- 03
Sign off.
The sign-off pack goes to the named approvers. The configuration they approved is the one that goes live, and it is snapshotted.
- 04
Monitor and re-test.
Real conversations are scored continuously. Any later change to a prompt, model or knowledge base re-runs the suite before release.
From production
We evaluate our own agents first
The same testing and scoring runs on the WhatsApp, email and voice agents we operate.
4
Voice agents we built and operate in GWC Logistics' contact centre in Qatar
- Test runs work on a disposable copy of the agent; production is never changed by a test
- Copy onlyTest runs work on a disposable copy of the agent; production is never changed by a test
- Dimensions every closed conversation is scored on, each with the judge's reasons
- 6Dimensions every closed conversation is scored on, each with the judge's reasons
- Every score stores the rubric version it was graded with
- VersionedEvery score stores the rubric version it was graded with
For compliance
Evidence your auditor can follow.
Every risk, test, result, approval and change is recorded, so your compliance team and auditors can follow what the agent was allowed to do, how it was tested and who approved it. We do not certify, and we make no compliance claim on your behalf. This works for agents we build and for agents you already run.
Best for
- ›CIOs asked to approve an agent they didn't build. You get a go-live gate with evidence behind it.
- ›Compliance and risk teams. You get criteria, results and approvals in one file.
- ›Heads of CX with an agent already live. You see how it is doing conversation by conversation, not through complaints.
- ›Teams about to change a model or a prompt. You see the scorecard before the switch, not the incidents after it.
FAQ
- How is this different from your agent-evaluation service?
- The agent-evaluation page explains how the tests and the scoring work. This programme puts them inside a governance process: a risk register, criteria agreed with the business, a go-live gate with named approvers, change control and an audit file.
- Can you evaluate agents we didn't build?
- Yes. If an agent is already live or about to launch, we can write its risk register and test suite, score a sample of its real conversations, and set up monitoring around it.
- Do you certify our agents as compliant?
- No. We are not a certification body and make no compliance claim on your behalf. We produce the record of risks, tests, results and approvals, so the people who audit you can see exactly how each agent was governed.
- What does the compliance team actually receive?
- A sign-off pack for each agent: the risks, the criteria, the test results with transcripts, the failures and their fixes, the exact configuration tested, and the approvals. After launch, they also get ongoing scores and the change log.
- What happens if an agent starts failing after launch?
- Scored conversations flag the problem with reasons, it goes into a review queue, and a fix is proposed. The fix is re-tested before release and counts as verified only when the flag stops appearing in new conversations. Earlier configurations can be restored from a snapshot.
- Where does the conversation data go?
- Our systems run on your own cloud and API accounts, so transcripts and scores stay in your estate under your access rules.
Put a go-live gate in front of every AI agent.
Book a free consult. Bring one agent, live or about to launch, and we will show you the risks we would test, the criteria we would propose, and what your sign-off pack would contain.