> ## Documentation Index > Fetch the complete documentation index at: https://docs.egma.ai/llms.txt > Use this file to discover all available pages before exploring further. # Tests and Runs: Simulate and Evaluate Voice Agents > Understand how Egma test files are structured, how runs execute them as live simulations, and what each of the four verdicts means. A **Test** describes one situation your voice agent should handle: who the caller is, what they want, and what the agent should do in response. A **Run** executes a set of tests against a connection, sending each one to your live agent as a real simulated conversation. Tests live as Markdown files in your repository. Runs are created in Egma and tracked through to a final verdict. ## Test anatomy Each test is a single Markdown file inside your `egma/tests/` folder. The file combines a YAML frontmatter block with prose sections that the simulator and grader read: ```markdown theme={null} --- name: missed-appointment-reschedule personas: [impatient-caller] version: tstv_01K… --- ## Scenario The caller missed yesterday's appointment and wants to reschedule this week. They are short on time and irritated. ## Expected behaviors 1. The agent acknowledges the missed appointment without blame. 2. The agent offers at least two concrete alternative slots. 3. The agent confirms the new booking before ending the call. ``` **YAML frontmatter fields** | Field | Required | Description | | ---------- | -------- | -------------------------------------------------------------------------------------------------------------------------------- | | `name` | Yes | A slug that identifies this test. Used as the filename and displayed in run results. | | `personas` | No | Which simulated caller persona(s) to use. Each persona produces one simulation per run. Leave it out to use the default persona. | | `version` | No | Written by `egma push` or `egma pull`. Do not edit by hand. | **Markdown sections** | Section | Required | Description | | ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `## Scenario` | Yes | Natural-language description of what the call is about. The simulator reads this prose to drive the conversation. | | `## Expected behaviors` | Yes | A numbered list of assertions. Each one becomes a judgment target for the grader. A test with no expected behaviors can never fail, so Egma will not accept one. | ### Mock tool overrides Your `egma/mock-tools.md` file defines the global mocked world: how Egma answers your agent's tool calls during simulations. This prevents tests from hitting your real backend and lets you control what branch the agent sees — an empty calendar, a failing service, a specific order state. ````markdown theme={null} ## Mock tools ### check_availability ```json { "answer": { "slots": [] }, "delay_ms": 250 } ```` ```` A test can override the project-level mock tools by adding the same section at the bottom of its own file. That override belongs to the test and is versioned with it. Use `error` instead of `answer` to simulate a tool that raises an exception: ```json { "error": "service_unavailable", "delay_ms": 500 } ```` ## Test versioning Every time you push a changed test to Egma, Egma mints a new `version_id` (`tstv_…`) and writes it back into your file's frontmatter. The previous version is never overwritten — it remains readable and is still referenced by any run that pinned it. This versioning guarantee means: **results from last week still say exactly what ran**. A green run from before a test change is still green, and it still names the scenario and expected behaviors that were green. `egma push` refuses to upload a test if Egma holds a newer version than your file knows about. Run `egma pull` first, review what changed, then push. ## Runs A Run starts a batch of simulations. You name the connection to use and the exact test versions to execute. Egma creates the run, queues one simulation per test-version × persona combination, and begins dispatching them to the simulator. ### Starting a run Make sure your local tests are uploaded and version IDs are in sync: ```bash theme={null} egma push ``` From your repository folder: ```bash theme={null} egma run ``` This pins the current version of every test, creates the run, and follows it live until the first verdict arrives. The terminal prints a `results_url` you can open immediately. The run continues after the CLI exits — closing your terminal never stops a run. To start a run non-interactively (for CI pipelines): ```bash theme={null} egma run --headless --no-follow ``` You can also use the API directly: ```bash theme={null} curl -X POST https://your-egma/api/runs \ -H "Authorization: Bearer egma_..." \ -H "Content-Type: application/json" \ -d '{ "connection": "con_01K…", "test_versions": ["tstv_01K…", "tstv_01K…"] }' ``` ### Simulation lifecycle Each simulation inside a run moves through a defined set of states: ``` queued → claimed → running → completed ↘ failed ↘ canceled ``` | Status | Meaning | | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `queued` | Waiting for the simulator to claim it. | | `claimed` | A simulator has taken the simulation and is preparing to start. | | `running` | The simulator is actively conducting the conversation. | | `completed` | The conversation finished normally. Awaiting grader judgment. | | `failed` | The simulator could not complete the conversation — the agent was unreachable or an unrecoverable error occurred. The grader still runs and writes `errored` verdicts. | | `canceled` | The run was stopped before this simulation reached a terminal state. | Execution and grading are two distinct phases. A simulation can be `completed` while judgment is still `pending`. The run results page shows both states separately so you always know which one you are waiting on. ### Verdicts After a simulation finishes, the grader reads the conversation and writes one verdict per expected behavior. Each verdict is one of four values — and Egma never collapses them: The agent did what the test expected. The behavior was observed in the conversation. The agent did not meet the expectation. Something in the agent's behavior needs attention. Nothing was judged. The test required a capability this connection does not have (for example, DTMF tones on a text connection), or the grader had nothing scoreable. The conversation could not be conducted — the agent was unreachable or Egma encountered a fault. The simulation attempted to run but did not complete successfully. `skipped` is not a pass. A test that could not run is reported honestly as skipped rather than silently counted as passing. Reporting it otherwise would send you hunting a bug that is not there. Each verdict carries its own rationale, the conversation turns it cites, and the grader that wrote it. ## Recordings Every voice simulation produces a **dual-channel audio recording** — the simulated caller on the left channel, your agent on the right. Recordings are stored in Egma's object store and are accessible from the run results page. The recording player appears directly beside the conversation transcript, so when a turn looks wrong you can listen to exactly what was spoken and heard — and confirm whether the agent misbehaved or the transcription did. Chat simulations and calls that never connected produce no recording. The player is shown only where a recording exists.