curl --request POST \
--url https://app.egma.ai/v1/grader-library/{graderDefinitionId}/use \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"scope": {
"simulations": [
{
"kind": "all"
}
],
"production": null
},
"passThreshold": 1
}
'{
"id": "<string>",
"projectId": "<string>",
"graderDefinitionId": "<string>",
"name": "<string>",
"description": "<string>",
"owner": "egma",
"type": "llm_as_judge",
"modalities": [
"chat"
],
"scopeEditable": true,
"removable": true,
"scope": {
"simulations": [
{
"kind": "all"
}
],
"production": {
"samplePercent": 50.5
}
},
"settings": {},
"passThreshold": 0.5,
"createdAt": "2023-11-07T05:31:56Z",
"updatedAt": "2023-11-07T05:31:56Z"
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}Use a grader in the current project
Adds an available definition with this project’s scope, settings, and pass threshold. These settings apply to future work; adding a grader does not change earlier simulation plans. A definition can be active only once per project. The example settings are for Response latency.
curl --request POST \
--url https://app.egma.ai/v1/grader-library/{graderDefinitionId}/use \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"scope": {
"simulations": [
{
"kind": "all"
}
],
"production": null
},
"passThreshold": 1
}
'{
"id": "<string>",
"projectId": "<string>",
"graderDefinitionId": "<string>",
"name": "<string>",
"description": "<string>",
"owner": "egma",
"type": "llm_as_judge",
"modalities": [
"chat"
],
"scopeEditable": true,
"removable": true,
"scope": {
"simulations": [
{
"kind": "all"
}
],
"production": {
"samplePercent": 50.5
}
},
"settings": {},
"passThreshold": 0.5,
"createdAt": "2023-11-07T05:31:56Z",
"updatedAt": "2023-11-07T05:31:56Z"
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}{
"error": "<string>",
"message": "<string>",
"details": {}
}Authorizations
An Egma API key.
Path Parameters
1Query Parameters
1Body
Select all simulations, particular test suites or tests, and/or a production sample from 1 through 100 percent. production: null disables production grading.
Show child attributes
Show child attributes
The minimum score for this grader's individual result to pass.
0 <= x <= 1Complete values for the definition's settingDefinitions. Omit settings to save the declared defaults on first use. LLM graders use llm_provider and llm_model; Response latency uses maximum_response_time_ms.
Response
The new current-project grader policy.
111egma, project llm_as_judge, code 1chat, voice Show child attributes
Show child attributes
0 <= x <= 1