Graders: Configure Judgment Criteria for Your Project

Project graders are the active judgment layer for a specific project. Each grader is a copy of an entry from the grader library — the entry supplies the prompt and the schema, and your copy holds the filled-in parameters, the scope it operates in, and whether its verdict can block a test run. When you press Use on a library entry in the dashboard, Egma calls POST /api/graders on your behalf. Use the API directly to automate grader configuration, provision graders during CI setup, or manage them programmatically across projects.

List graders

Returns the running graders for a project, ordered newest first. GET /api/graders

string

The prj_ identifier of the project to read. Omit this when your API key is already scoped to a project.

string

Pagination cursor. Pass the next_cursor value from a previous response to fetch the next page. Omit to start from the newest grader.

array

Array of grader objects. See the grader object reference below.

string | null

Cursor for the next page. null means you have reached the last page.

List graders

curl https://your-egma-instance/api/graders \
  -H "Authorization: Bearer egma_sk_..."

Filter by project

curl "https://your-egma-instance/api/graders?project=prj_abc123" \
  -H "Authorization: Bearer egma_sk_..."

Create a grader

Activates a library entry for your project by creating a copy with the parameters you supply. POST /api/graders

string

required

The grl_ identifier of the library entry to copy. Read the grader library to see available entries and the parameters each one accepts.

object

The library entry’s form filled in. Shape varies by entry — for example, a latency grader accepts { "metric": "turn_response_latency", "bound": 2000 }. Read the library entry’s params field to see what is required. Omit entirely for entries that ask for nothing.

string

Where this grader judges. One of "simulations", "production", or "all". Defaults to "all" when omitted.

boolean

Whether a failing verdict from this grader can block a test run from passing. Set to false to make the grader diagnostic-only — it reports, but never fails anything. Defaults to true.

string

A display name for this copy. Defaults to the library entry’s own name when omitted.

string

A note your team leaves on this grader, explaining why it is switched on.

number

Percentage of live production traffic this grader judges, as a whole number between 0 and 100. Only meaningful when scope includes production.

string

The prj_ identifier of the project to create the grader in. Omit when your API key is already scoped to a project.

Returns 201 Created with the new grader object.

string

The grd_ identifier for this grader copy.

string

The library entry this copy was made from.

number

The current version number. Starts at 1 and increments each time params are changed.

string

The identifier of the current version. Verdicts reference this to record exactly which parameters decided them.

object

The filled-in parameter values for this version.

Create a grader

curl -X POST https://your-egma-instance/api/graders \
  -H "Authorization: Bearer egma_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "library_id": "grl_latency_01",
    "params": { "metric": "turn_response_latency", "bound": 1500 },
    "scope": "all",
    "required": true,
    "name": "Response latency (1.5 s)"
  }'

The params you send are validated against the library entry’s schema. Sending a parameter the entry never asked for, or a value outside the allowed range, returns 422 Unprocessable Content with a message naming what went wrong.

Update a grader

Modifies a running grader. What you send determines whether Egma mints a new version or updates in place. PATCH /api/graders/:graderId

string

required

The grd_ identifier of the grader to update.

object

New parameter values. Sending this field mints a new version — all verdicts already written remain readable under the version that decided them.

string

Updated scope. Written in place; does not change existing verdicts.

boolean

Updated blocking flag. Written in place. Because the outcome fold runs at read time, changing required from true to false immediately re-reads runs that previously failed on this grader alone as passing.

string

Updated display name. Written in place.

string | null

Updated note. Send null or an empty string to clear the existing note. Omit to leave it unchanged.

number

Updated production sample rate. Written in place.

string

The prj_ identifier of the project. Omit when your key is already scoped to a project.

Returns the updated grader object.

Sending params mints a new version of the grader. Fields like scope, required, name, and description are written in place and never create a new version. Egma decides which rule applies — you send the full body and read the version number back.

You cannot change a grader’s library_id. A grader’s type is set by the entry it was copied from, and every historical version was shaped by that type. To judge with a different entry, create a new grader from it and delete the old one.

Change parameters (mints new version)

curl -X PATCH https://your-egma-instance/api/graders/grd_abc123 \
  -H "Authorization: Bearer egma_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "params": { "metric": "turn_response_latency", "bound": 2000 }
  }'

Update scope and required flag (in-place)

curl -X PATCH https://your-egma-instance/api/graders/grd_abc123 \
  -H "Authorization: Bearer egma_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "scope": "simulations",
    "required": false
  }'

Delete a grader

Switches a grader off. From the moment this returns, no new conversations are judged by this copy. DELETE /api/graders/:graderId

string

required

The grd_ identifier of the grader to delete.

string

The prj_ identifier of the project. Omit when your key is scoped to a project.

Returns the deleted grader’s id, name, and deleted_at timestamp.

Delete a grader

curl -X DELETE https://your-egma-instance/api/graders/grd_abc123 \
  -H "Authorization: Bearer egma_sk_..."

Deletion is a soft delete. The grader’s row is marked as deleted, and its versions are kept intact. Every verdict already written remains readable and still references the version that decided it — an old run keeps its full meaning even after the grader that produced it is switched off. The underlying library entry is also retained, because historical verdicts need it to stay interpretable.

The grader object

string

The grd_ identifier for this grader copy.

string

The grl_ identifier of the library entry this copy was made from.

string

The project this grader belongs to.

string

Display name for this copy.

string | null

Optional note from your team.

string

The kind of judgment this grader makes, inherited from the library entry.

boolean

Whether a failing verdict from this grader blocks a run from passing.

string

Where this grader judges — "simulations", "production", or "all".

number

Percentage of live production traffic this grader judges.

number

Current version number, starting at 1.

string

Identifier of the current version. Referenced by verdicts to record which parameters decided them.

object

The filled-in parameter values active for this version.

string

ISO 8601 timestamp of when this grader was created.

string

ISO 8601 timestamp of the last update.

Overview

Agents & Tests

Graders

Telemetry

Account

Graders: Configure Judgment Criteria for Your Project

List graders

Create a grader

Update a grader

Delete a grader

The grader object

​List graders

​Create a grader

​Update a grader

​Delete a grader

​The grader object

List graders

Create a grader

Update a grader

Delete a grader

The grader object