Skip to main content
POST
Create and use a custom LLM grader

Authorizations

Authorization
string
header
required

An Egma API key.

Query Parameters

projectId
string
Minimum string length: 1

Body

application/json
name
string
required
gradingInstructions
string
required

One rule to decide and the conversation evidence to inspect.

passesWhen
string
required

The evidence that makes the rule pass. Include how to handle a conversation where the checked action never occurs.

failsWhen
string
required

The evidence that makes the rule fail.

scope
object
required

The future simulations and/or production sample this project should grade.

passThreshold
number
required

The minimum score for this grader's individual result to pass.

Required range: 0 <= x <= 1
description
string | null
settings
object

A complete llm_provider and llm_model pair from Get grader model choices. Omit settings to use the declared defaults.

Response

The custom definition and its current-project policy.

definition
object
required
grader
object
required