The tabs
How scoring works
Every reply is graded on the live path for unsupported claims, with no model call, so it costs no latency. See Guardrails. Beyond that, conversations are evaluated against a rubric covering whether the customer’s question was answered, whether the agent followed your rules, tone and persona fidelity, and whether the conversation reached a sensible outcome. Scores are aggregated so you see distributions rather than a single blended number. A workspace with a 4.2 average made up of mostly 5s and a tail of 1s has a different problem from one that is uniformly mediocre.Signals that feed quality
Automatic grading
Every turn, on the live path plus an asynchronous deeper pass.
Operator feedback
Thumbs up and down from your team, with a reason. See Explainability.
Customer ratings
CSAT, which is the only signal that comes from outside.
Outcomes
Whether conversations resolved, escalated, or were abandoned.
Calibration
The platform also tracks whether the agent’s confidence matches its accuracy. An agent that hedges when it is right and asserts when it is wrong is poorly calibrated even with an acceptable hallucination rate, and it erodes trust faster than occasional errors do.Closing the loop
Quality data is not just a report.- Failures feed knowledge gaps. A cluster of bad answers on one topic becomes a ranked, actionable finding.
- Corrections teach categories. Fixing a wrong category is training data.
- Prompt changes can be tested rather than guessed. Run two variants against real traffic on a split and compare their scores before committing.
- Releases can be compared and rolled back. See The agent.
Before you go live and after
Prompt and configuration changes are evaluated against recorded conversations before they reach customers, so a change that would have made past conversations worse is caught first. This is how a change gets promoted rather than merely deployed.Reading the numbers honestly
- Look at the distribution, not the average.
- Compare like with like. Channel mix and volume changes move quality numbers on their own.
- Where CSAT and QA disagree, read the conversation. The platform’s opinion of its own work and the customer’s opinion diverging is the most informative thing in the report.
- A rising handoff rate is not automatically bad. An agent that escalates more because your rules got stricter is working correctly.