An agent that passed the demo can still promise a price it shouldn't.
We test every change to an agent against scripted customers before it ships, and score every real conversation after it runs — WhatsApp, email and voice — so you know how your agent is doing from evidence, not from the complaints that reach you.
What we do
Six parts of an evaluation that changes what ships
Pre-production test suites
Scenarios written as real customers — the one who pushes for a price, the one who demands a human, the one who switches language — each graded against pass/fail criteria.
Model and prompt benchmarking
Every candidate model and prompt runs through the same suite, with pass rate, task success, time per turn and reply length side by side. A model switch becomes a scorecard decision.
Production scoring on every channel
WhatsApp, email and voice conversations scored on resolution, accuracy, qualification, handover, tone and efficiency — with the judge's reasons, not just a number.
CRM capture checks
For voice, every field the agent said it recorded is checked against the CRM record as it actually stands. Measured, not judged.
Calibrated, versioned rubrics
The judge is told what you publish, so an approved figure isn't called a fabrication. Every score stores the rubric version it was graded with, so old and new scores are never mixed.
A backlog that proves fixes held
Each problem becomes a concrete fix — a knowledge gap, an instruction, a handover rule. Once actioned, it is marked verified only when its flag stops recurring in new conversations.
How it works
Test before it ships. Score after it runs.
- 01
Write the suite
Scripted customers, pass/fail criteria for each, and the variants to compare. The suite lives in the repository next to the prompts it grades.
- 02
Run it on a clone
The harness copies the live agent, applies each variant to the copy, plays every scenario, grades every transcript and deletes the copy.
- 03
Promote the winner
The scorecard shows pass rate, task success, time per turn and every failure with its reason. The old configuration is snapshotted before a promote.
- 04
Score production
Every real conversation is graded on six dimensions, with measured facts alongside. Flags fill a review queue; fixes fill a backlog.
From production
What the harness measured for GWC
Figures from the stored scorecards behind GWC's voice agents. Every run is kept, so each number traces back to the transcripts behind it.
- Rubric pass rate of the rewritten Arabic agent across 60 graded checks, against 78% for the prompt it replaced
- 100%Rubric pass rate of the rewritten Arabic agent across 60 graded checks, against 78% for the prompt it replaced
- Criteria every outbound call is graded on — from saying the call is recorded to writing the exact CRM field keys
- 14Criteria every outbound call is graded on — from saying the call is recorded to writing the exact CRM field keys
- Changes a test run makes to the production agent — every run works on a disposable clone
- 0Changes a test run makes to the production agent — every run works on a disposable clone
Why us
We grade our own agents before anyone else does.
Most agent projects ship on a good demo and find out the rest in production. Ours don't: the same harness and the same judge run on the WhatsApp, email and voice agents we operate for GWC. It is how we decide which model to run, which prompt to keep and which answer to fix next — and it is what we hand over to you.
Best for
- ›Teams with an agent already live. You want to know how it is really doing, one conversation at a time.
- ›Teams about to change a model or a prompt. You want the scorecard before the switch, not the complaints after it.
- ›Brands where one wrong answer is expensive. A record of what the agent said, and why it was judged right or wrong.
- ›Bilingual and multi-dialect deployments. Language, dialect and register graded as criteria, not hoped for.
Know how your agent is doing. Conversation by conversation.
Book a 30-minute call. Bring a sample of real transcripts and we'll show you what a judge would flag — and what we would test before your next change ships.