0 to 1, or no score when it cannot grade
the evidence. A completed grade includes that score, its details, and the pass
threshold selected for the conversation.
Read an individual result
Suppose Expected behaviors checks four statements. Three pass, so the grader
score is
0.75. With a pass threshold of 1, that grader fails. With a
threshold of 0.75, it passes. The failed statement remains visible in either
case.
For LLM graders, a cannot_determine decision counts as 0, just like
not_met, and remains in the total number of criteria. The original decision
and reason remain visible in the grade details. For example, one met criterion,
one not met, and two undetermined criteria score 0.25 and fail a threshold
of 1.
A grading error means Egma could not produce a valid result, such as when a
model request fails or returns an incomplete response. It has no score. An
undetermined decision in a valid response is a completed, scored result.
Read the combined score
When every selected grader has a score, the combined score is their unweighted arithmetic mean. Scores of1 and 0.5 produce a combined score of
0.75.
This value helps compare conversations. It has no overall pass threshold.
Egma does not turn it into a pass/fail result for a simulation, test, suite, or
run. Read the individual grades to see which checks passed.
The combined score is unavailable when no grader was selected, grading is still
pending, or a selected grader has no score.
Separate progress from quality
Grading state describes whether work has finished:complete does not guarantee that all agent evidence arrived. Grading can
proceed with the available evidence when the wait for agent evidence ends.
Check the simulation’s agentPovIncomplete field and
review the evidence source
when a result appears to omit something the agent did.
A simulation that fails or is canceled before producing a completed trace has
no grading work to wait for. Its API gradingState is null.