Objective
Understand what the evaluator graded, which scorecard version it used, and the evidence behind each criterion.
Before you begin
You need Quality: Manage or View Only to open company-wide raw grades, and a completed run. The original AI result is an unpublished draft until a manager shares it with the agent.
Resolution
- Open the run. In QA Analysis · QA Grades, find the ticket and expand its row, then open the full run page. The run records its category, origin, status, and frozen scorecard version.
- Select the graded target. Read the overall ticket result or an agent result as labeled. An agent batch grades only its selected agent and has no Overall ticket grade. Other runs may show both overall and agent targets.
- Read the summary. Check the effective numeric score, applicable criterion count, and critical result. A critical failure is separate from the percentage score. N/A criteria do not contribute points; remaining applicable weights are renormalized.
- Inspect criteria. Expand a group and criterion to read its standard, AI outcome, earned weight or insight, confidence when available, and any manager decision. The page uses the scorecard definition frozen with the evaluation, even if the current scorecard has changed.
- Check cited evidence. Open the criterion's evidence to see the conversation excerpts or connected evidence it cited, then compare them with the transcript and ticket context. A criterion may cite no evidence; treat that as a reason to review the conclusion carefully.
The run page places ticket history beside the grade summary and criterion outcomes. Customer and agent names, the email, ticket ID, and order number are generalized in this example. - Separate result from publication. A reviewer can correct a criterion grade before publishing the result to the linked agent. See How Scorecards, Evaluations, Reviews, And Coaching Fit Together for the downstream workflow.
If the page shows a skip or failure instead of a grade, use Investigate A Skipped Or Failed Evaluation.