Parts of a test
Scenario
The scenario tells the caller what they want and how to act in this conversation. Describe one situation in plain language. Keep unrelated situations in separate tests. For example - a scneario could be ‘You have an upcoming appointment and want to change it’Expected behaviors
Expected behaviors are observable requirements for your agent. Include at least one. The Expected behaviors grader checks these statements against the simulation’s evidence. For example: “The agent asks the caller to confirm the appointment date and time before booking.” Use one clear claim per behavior.Personas
A persona defines the kind of caller. The scenario defines what that caller wants in this test. Select one or more personas. Each selected persona adds a simulation when you run the suite. Typically personas represet callers with differnt personalities like interruptive, angry or they could talk in a different language like Spanish etc.Mock tools
Mock tools provide a fixed response or error for a named tool. For example, make an availability tool return no appointment slots. A tool without a mock runs its real implementation. See Mock tool responses for examples and supported connections.Env
Env represents startup context. These fields are passed to the connection before the conversation starts. For LiveKit,job_dispatch_metadata supplies the JSON data your worker reads from ctx.job.metadata. For Retell, retell_dynamic_variables supplies string values.
This is the Env field in the UI and the ## Env section in a test file. It does not set shell environment variables or store credentials.
Create a test
Follow Create a test for UI steps and CLI commands.Test suite
Each test is part of a test suite by default.Run your tests
Running a test means running the test suite associated with it. A run executes the complete suite against one agent connection. Each test and selected persona combination produces a simulation. Two tests with two personas each produce four simulations. Start a run, then inspect each simulation’s conversation and grading results. The same suite can run against different connections, so you can check a flow through both text and voice.Keep your tests current
Add tests as you add flows and capabilities to your agent. Keep tests for working behavior in a regression suite, and use a separate suite for behavior you are still improving. If you edit tests in both the UI and your repository, runegma pull before local edits and egma push when they are ready. See Test suites for synchronization and conflict handling.