What is a grader?

A grader checks conversation evidence and assigns a score from 0 to 1. A grade records that score, the supporting details, and the grader’s pass threshold. A score at or above the threshold passes that grader’s check. A metric records an observed fact, such as response latency. A grader can use that metric to decide whether the conversation meets a requirement.

​

Add or update a grader

Add a grader from the library, or create a custom grader for your own rules. Update a grader to change its scope and settings.

​

Choose a grader

Read Types of graders to choose between language model checks and code-based checks.

​

Choose where a grader applies

Add graders to your project and configure their scope, settings, and pass thresholds. A scope can select simulations or a sample of production conversations. Expected behaviors always grades simulations and does not grade production conversations. Read Configure graders for the available scope controls and regrading.

​

Read each result

A grader can fail to produce a score when evidence is missing or grading fails. Check its state and evidence. A missing score is not a pass. Each grader has its own result. The combined score is the mean of the selected grader scores and does not decide an overall pass or fail. See Scores and thresholds.