> ## Documentation Index > Fetch the complete documentation index at: https://docs.roark.ai/llms.txt > Use this file to discover all available pages before exploring further. # Run a simulation > Starts a simulation and returns the run. Send `template` to run one of the built-in templates: it supplies the metrics and checks, and for some templates the flows too, so the request only names the agent and the direction. Send `plan` to describe a simulation yourself and run it once. Send `planId` to run a plan you already have. `template` and `plan` both resolve to a run plan, returned as `simulationRunPlanId`. Add `saveAsPlan` to keep it, or read it back to see exactly what ran. A plan built from a template is a snapshot: retuning the template later never changes what that plan runs, which is what makes a saved one safe to pin in CI. ## OpenAPI ````yaml /api-reference/openapi.documented.json post /v1/simulation/run openapi: 3.1.0 info: title: Roark Analytics API description: >- The Roark Analytics API gives you access to the same API that powers the award winning Roark Analytics platform. version: 1.0.0 servers: - description: Production url: https://api.roark.ai security: - Bearer: [] tags: - name: Agent - name: Agent Endpoint - name: Call - name: Chat - name: Metric - name: Metric Policy - name: Metric Collection Job - name: Customer Flow - name: Customer Flow Edge Case - name: Simulation - name: Simulation Persona - name: Simulation Environment - name: Simulation Scenario - name: Simulation Run Plan - name: Simulation Template - name: Simulation Run Plan Job - name: Simulation Job - name: HTTP Request Definition - name: Webhook - name: Issue - name: Knowledge Base - name: Config - name: CLI Auth - name: Call Analysis - name: Health paths: /v1/simulation/run: post: tags: - Simulation summary: Run a simulation description: >- Starts a simulation and returns the run. Send `template` to run one of the built-in templates: it supplies the metrics and checks, and for some templates the flows too, so the request only names the agent and the direction. Send `plan` to describe a simulation yourself and run it once. Send `planId` to run a plan you already have. `template` and `plan` both resolve to a run plan, returned as `simulationRunPlanId`. Add `saveAsPlan` to keep it, or read it back to see exactly what ran. A plan built from a template is a snapshot: retuning the template later never changes what that plan runs, which is what makes a saved one safe to pin in CI. operationId: postV1SimulationRun requestBody: content: application/json: schema: $ref: '#/components/schemas/RunSimulationInput' responses: '200': description: The run that was started content: application/json: schema: type: object properties: data: $ref: '#/components/schemas/RunSimulationResponse' required: - data '400': content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' description: Validation error example: type: validation code: invalid_parameter message: The request was invalid param: email description: Bad Request '401': content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' description: Authentication error example: type: authentication code: unauthorized message: Authentication required description: Unauthorized '402': content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' description: Insufficient credit example: type: payment_required code: insufficient_credits message: >- Not enough credit to start this work. Add credit and try again. description: Payment Required '403': content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' description: Permission error example: type: forbidden code: permission_denied message: You do not have permission to access this resource description: Forbidden '404': content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' description: Not found error example: type: not_found code: resource_not_found message: The requested resource could not be found description: Not Found '429': content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' description: Rate limit error example: type: rate_limit code: too_many_requests message: Rate limit exceeded description: Too Many Requests '500': content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' description: Server error example: type: internal code: internal_error message: Internal server error description: Internal Server Error x-codeSamples: - lang: JavaScript source: |- import Roark from '@roarkanalytics/sdk'; const client = new Roark({ bearerToken: process.env['ROARK_API_BEARER_TOKEN'], // This is the default and can be omitted }); const response = await client.simulation.run({ plan: { agentEndpoints: [{ id: '182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e' }], direction: 'INBOUND', maxSimulationDurationSeconds: 300, metrics: [{}], }, }); console.log(response.data); - lang: Python source: |- import os from roark_analytics import Roark client = Roark( bearer_token=os.environ.get("ROARK_API_BEARER_TOKEN"), # This is the default and can be omitted ) response = client.simulation.run( plan={ "agent_endpoints": [{ "id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e" }], "direction": "INBOUND", "max_simulation_duration_seconds": 300, "metrics": [{}], }, ) print(response.data) components: schemas: RunSimulationInput: oneOf: - $ref: '#/components/schemas/RunSimulationFromConfig' - $ref: '#/components/schemas/RunSimulationFromPlanId' - $ref: '#/components/schemas/RunSimulationFromTemplate' description: >- A built-in template to run, a simulation to configure and run, or the id of a plan to run. RunSimulationResponse: type: object properties: simulationRunPlanJobId: type: string format: uuid description: The run. Poll it with GET /v1/simulation/plan/job/{jobId}. status: type: string enum: - PENDING - QUEUED - CREATING_SNAPSHOTS - CREATING_SIMULATIONS - PREPARING_CAPACITY - RUNNING_SIMULATIONS - COMPLETED - FAILED - TIMED_OUT - CANCELLED - CANCELLING - ENDING_SIMULATIONS description: >- Initial status. PENDING normally, or QUEUED when the plan runs sequentially and another job of its is still active. createdAt: type: string description: When the run was created, ISO 8601. simulationRunPlanId: type: string format: uuid description: >- The run plan behind this run, present whether or not it was saved. Pass it back as `planId` to run the same configuration again. savedAsPlan: type: boolean description: >- Whether that plan is listed by GET /v1/simulation/plan. False for an unsaved run, whose plan is hidden. simulationJobCount: type: integer description: How many simulated calls this run places. required: - simulationRunPlanJobId - status - createdAt - simulationRunPlanId - savedAsPlan - simulationJobCount description: A started simulation run. ErrorResponse: type: object properties: type: type: string enum: - validation - authentication - forbidden - not_found - conflict - payment_required - rate_limit - internal description: The error type category examples: - validation - authentication code: type: string description: Machine-readable error code identifier examples: - invalid_parameter - missing_required_field - unauthorized message: type: string description: Human-readable error message examples: - The request was invalid - Authentication required param: type: string description: The parameter that caused the error (if applicable) examples: - email - user_id details: description: Additional error context information required: - type - code - message RunSimulationFromConfig: type: object properties: plan: $ref: '#/components/schemas/InlineRunPlanConfig' saveAsPlan: type: boolean description: >- Keeps this configuration as a run plan, listed by GET /v1/simulation/plan and re-runnable with `planId`. Requires `plan.name`, since a plan you meant to keep should not be filed under a generated one. Omitted or false gives a one-off. The run still needs a plan to execute, so one is created either way, but it is hidden: it carries this run and nothing else. variables: anyOf: - type: object additionalProperties: type: string description: >- Global format: key-value pairs that apply to ALL scenarios in the plan example: orderNumber: '12345' environment: staging - type: array items: type: object properties: flowId: type: string format: uuid description: A customer flow this plan runs. happyPath: type: boolean const: true description: Narrow to the flow's happy path. edgeCaseId: type: string format: uuid description: Narrow to one edge case of that flow. variables: type: object additionalProperties: type: string description: The values to apply. required: - flowId - variables description: Values scoped to the flows this plan runs example: - flowId: 550e8400-e29b-41d4-a716-446655440000 variables: orderNumber: '12345' - flowId: 550e8400-e29b-41d4-a716-446655440000 happyPath: true variables: orderNumber: '55555' - flowId: 550e8400-e29b-41d4-a716-446655440000 edgeCaseId: 7a3d2e1f-c4b5-6a89-0d1e-2f3a4b5c6d7e variables: orderNumber: '67890' - type: array items: type: object properties: scenarioId: type: string format: uuid description: ID of the scenario to apply variables to variables: type: object additionalProperties: type: string description: Key-value pairs for this scenario required: - scenarioId - variables deprecated: true description: >- Deprecated, for plans built on scenarios. Plans built on customer flows target them with `flowId` instead. example: - scenarioId: 550e8400-e29b-41d4-a716-446655440000 variables: orderNumber: '12345' - scenarioId: 7a3d2e1f-c4b5-6a89-0d1e-2f3a4b5c6d7e variables: orderNumber: '67890' description: >- Values for the {{variables}} the run resolves, overriding whatever the plan has pinned. An object applies them to the whole run: { "orderNumber": "12345", "tier": "gold" } An array applies them per flow, or to just its happy path or one of its edge cases, when a single set will not do. Each entry carries what it applies to: [ { "flowId": "550e8400-...", "variables": { "orderNumber": "12345" } }, { "flowId": "550e8400-...", "happyPath": true, "variables": { "orderNumber": "55555" } }, { "flowId": "550e8400-...", "edgeCaseId": "7a3d2e1f-...", "variables": { "orderNumber": "67890" } } ] An entry that narrows to neither covers everything that flow resolves. A flow this plan does not attach, or an edge case that does not belong to the flow, is rejected rather than ignored. A plan built on scenarios rather than customer flows targets them the same way, with `scenarioId` in place of `flowId`. That form is deprecated alongside scenarios themselves, and still accepted so runs against those plans keep working. required: - plan additionalProperties: false title: New simulation description: >- Describe a simulation and run it, optionally keeping the configuration as a reusable plan. example: saveAsPlan: true plan: name: Billing regression direction: OUTBOUND maxSimulationDurationSeconds: 300 agentEndpoints: - id: 7c9e6679-7425-40de-944b-e07fc1f90ae7 metrics: - slug: customer_satisfaction flows: - id: 550e8400-e29b-41d4-a716-446655440000 happyPath: true edgeCases: ALL variables: orderNumber: '12345' RunSimulationFromPlanId: type: object properties: planId: type: string format: uuid description: >- The run plan to run, saved or hidden. Rename or unhide it with PUT /v1/simulation/plan/{planId}. variables: anyOf: - type: object additionalProperties: type: string description: >- Global format: key-value pairs that apply to ALL scenarios in the plan example: orderNumber: '12345' environment: staging - type: array items: type: object properties: flowId: type: string format: uuid description: A customer flow this plan runs. happyPath: type: boolean const: true description: Narrow to the flow's happy path. edgeCaseId: type: string format: uuid description: Narrow to one edge case of that flow. variables: type: object additionalProperties: type: string description: The values to apply. required: - flowId - variables description: Values scoped to the flows this plan runs example: - flowId: 550e8400-e29b-41d4-a716-446655440000 variables: orderNumber: '12345' - flowId: 550e8400-e29b-41d4-a716-446655440000 happyPath: true variables: orderNumber: '55555' - flowId: 550e8400-e29b-41d4-a716-446655440000 edgeCaseId: 7a3d2e1f-c4b5-6a89-0d1e-2f3a4b5c6d7e variables: orderNumber: '67890' - type: array items: type: object properties: scenarioId: type: string format: uuid description: ID of the scenario to apply variables to variables: type: object additionalProperties: type: string description: Key-value pairs for this scenario required: - scenarioId - variables deprecated: true description: >- Deprecated, for plans built on scenarios. Plans built on customer flows target them with `flowId` instead. example: - scenarioId: 550e8400-e29b-41d4-a716-446655440000 variables: orderNumber: '12345' - scenarioId: 7a3d2e1f-c4b5-6a89-0d1e-2f3a4b5c6d7e variables: orderNumber: '67890' description: >- Values for the {{variables}} the run resolves, overriding whatever the plan has pinned. An object applies them to the whole run: { "orderNumber": "12345", "tier": "gold" } An array applies them per flow, or to just its happy path or one of its edge cases, when a single set will not do. Each entry carries what it applies to: [ { "flowId": "550e8400-...", "variables": { "orderNumber": "12345" } }, { "flowId": "550e8400-...", "happyPath": true, "variables": { "orderNumber": "55555" } }, { "flowId": "550e8400-...", "edgeCaseId": "7a3d2e1f-...", "variables": { "orderNumber": "67890" } } ] An entry that narrows to neither covers everything that flow resolves. A flow this plan does not attach, or an edge case that does not belong to the flow, is rejected rather than ignored. A plan built on scenarios rather than customer flows targets them the same way, with `scenarioId` in place of `flowId`. That form is deprecated alongside scenarios themselves, and still accepted so runs against those plans keep working. required: - planId additionalProperties: false title: Existing plan description: Run a plan that already exists. example: planId: 3a1d5e7c-9b2f-4a6d-8c31-5f7e9d0a2b4c variables: orderNumber: '99887' RunSimulationFromTemplate: type: object properties: template: type: string minLength: 1 description: The template to run, as listed by GET /v1/simulation/template. example: health-check agentEndpoints: type: array items: type: object properties: id: type: string format: uuid required: - id minItems: 1 description: The agent endpoints to call. No template can know these. direction: type: string enum: - INBOUND - OUTBOUND description: Direction of the simulation (INBOUND or OUTBOUND) example: INBOUND flows: type: array items: $ref: '#/components/schemas/RunPlanFlowSelection' minItems: 1 description: >- The flows to run, in the same shape a run plan takes them. Required when the template lists no flows of its own: it presets what to measure, and this says what to measure it on. Optional when it does, where these REPLACE the ones it would have run, so you can narrow a suite to the cases you care about. Either way, GET /v1/simulation/template lists the flows and variant ids each template covers. example: - slug: sf-prompt-injection edgeCases: - slug: data-embedded-injection - id: 3a1d5e7c-9b2f-4a6d-8c31-5f7e9d0a2b4c happyPath: true name: type: string minLength: 1 description: >- What to call this. Defaults to the template's name and the date, and required with `saveAsPlan`. saveAsPlan: type: boolean description: >- Keeps the resolved configuration as a run plan, listed by GET /v1/simulation/plan and re-runnable with `planId`. Requires `name`. iterationCount: type: integer minimum: 1 maximum: 10000 default: 1 description: Number of iterations to run for each test case (1-10000) example: 1 maxConcurrentJobs: type: integer minimum: 1 default: 5 description: Maximum number of concurrent simulation jobs example: 5 maxSimulationDurationSeconds: type: integer minimum: 1 description: >- Defaults to the template's `defaultMaxSimulationDurationSeconds`, as returned by GET /v1/simulation/template. silenceTimeoutSeconds: type: integer minimum: 1 default: 30 description: Timeout in seconds for silence detection example: 30 endCallPhrases: type: array items: type: string default: - goodbye description: Phrases that trigger end of call. Empty array disables the feature. example: - goodbye endCallReasons: type: array items: type: string description: >- Semantic conditions that trigger end of call. The LLM evaluates the conversation against these conditions. Defaults to the template's `defaultEndCallReasons`, as returned by GET /v1/simulation/template. Pass an empty array to run with none. example: - Order has been confirmed by the agent executionMode: type: string enum: - PARALLEL - SEQUENTIAL_SAME_RUN_PLAN - SEQUENTIAL_PROJECT default: PARALLEL description: Execution mode (PARALLEL or SEQUENTIAL) example: PARALLEL enrichWithLiveConversation: type: boolean default: false description: >- Merge the customer's own recording of the real call into each simulation, so metrics can be scored against the live leg as well as the simulated one. This is the API equivalent of the dashboard's live-enrichment toggle. With this on, the run provisions a phone number and holds each call open for up to 15 minutes waiting for a matching call to be posted to POST /v1/call. A call matches on the provisioned number (`roarkPhoneNumber` on the job) with a start time inside the simulation window. If nothing arrives, the simulation still completes and any `LIVE`-sourced metric produces no value. Required by any metric whose `requiresLiveConversation` is true: without it that metric is silently skipped. example: false variables: anyOf: - type: object additionalProperties: type: string description: >- Global format: key-value pairs that apply to ALL scenarios in the plan example: orderNumber: '12345' environment: staging - type: array items: type: object properties: flowId: type: string format: uuid description: A customer flow this plan runs. happyPath: type: boolean const: true description: Narrow to the flow's happy path. edgeCaseId: type: string format: uuid description: Narrow to one edge case of that flow. variables: type: object additionalProperties: type: string description: The values to apply. required: - flowId - variables description: Values scoped to the flows this plan runs example: - flowId: 550e8400-e29b-41d4-a716-446655440000 variables: orderNumber: '12345' - flowId: 550e8400-e29b-41d4-a716-446655440000 happyPath: true variables: orderNumber: '55555' - flowId: 550e8400-e29b-41d4-a716-446655440000 edgeCaseId: 7a3d2e1f-c4b5-6a89-0d1e-2f3a4b5c6d7e variables: orderNumber: '67890' description: >- Values for the {{variables}} the run resolves. An object applies them everywhere; an array targets a flow, its happy path, or one of its edge cases with `flowId`. The scenario-scoped form the other variants accept is not valid here: a template run is always flow-based, so there would be no scenario for it to reach. required: - template - agentEndpoints - direction additionalProperties: false title: From template description: Run one of the built-in templates against your agent. example: template: health-check direction: OUTBOUND agentEndpoints: - id: 7c9e6679-7425-40de-944b-e07fc1f90ae7 InlineRunPlanConfig: type: object properties: name: type: string minLength: 1 description: >- What to call this. Generated from the date when omitted, and required with `saveAsPlan`. example: Billing regression description: type: string description: Description of the run plan example: A run plan for testing inbound calls direction: type: string enum: - INBOUND - OUTBOUND description: Direction of the simulation (INBOUND or OUTBOUND) example: INBOUND iterationCount: type: integer minimum: 1 maximum: 10000 default: 1 description: Number of iterations to run for each test case (1-10000) example: 1 maxConcurrentJobs: type: integer minimum: 1 default: 5 description: Maximum number of concurrent simulation jobs example: 5 maxSimulationDurationSeconds: type: integer minimum: 1 description: Maximum duration in seconds for each simulation example: 300 silenceTimeoutSeconds: type: integer minimum: 1 default: 30 description: Timeout in seconds for silence detection example: 30 endCallPhrases: type: array items: type: string default: - goodbye description: Phrases that trigger end of call. Empty array disables the feature. example: - goodbye endCallReasons: type: array items: type: string default: [] description: >- Semantic conditions that trigger end of call. The LLM evaluates the conversation against these conditions. Empty array disables the feature. example: - Order has been confirmed by the agent executionMode: type: string enum: - PARALLEL - SEQUENTIAL_SAME_RUN_PLAN - SEQUENTIAL_PROJECT default: PARALLEL description: Execution mode (PARALLEL or SEQUENTIAL) example: PARALLEL scenarios: type: array items: type: object properties: id: type: string format: uuid description: Scenario ID variables: type: object additionalProperties: type: string description: >- Template variables for this scenario instance. The same scenario can appear multiple times with different variables. example: customerName: John Doe appointmentDate: '2024-02-15' required: - id minItems: 1 deprecated: true description: >- Deprecated: use `flows` instead. Scenarios to include in this run plan. The same scenario ID can appear multiple times with different variables. flows: type: array items: $ref: '#/components/schemas/RunPlanFlowSelection' minItems: 1 description: >- Customer flows to include in this run plan. The same flow can appear more than once with a different persona override or different variables. personas: type: array items: type: object properties: id: type: string format: uuid required: - id minItems: 1 description: >- Personas to include in this run plan. Required with `scenarios`; ignored with `flows`, where each variant carries its own persona. agentEndpoints: type: array items: type: object properties: id: type: string format: uuid required: - id minItems: 1 description: Agent endpoints to include in this run plan metrics: type: array items: $ref: '#/components/schemas/RunPlanMetricRef' minItems: 1 description: >- Metric definitions to include in this run plan. Reference each by `id` (UUID) or `slug`. includeFlowMetrics: type: boolean default: true description: >- Also collect each attached flow's own metrics, on top of the `metrics` named here. Default true, which is what you want when you brought your own flows and their graders. Set false for a run whose metric list is meant to be exhaustive: a template like Load Testing or Voicemail deliberately grades a narrow set, and inheriting every flow metric on top multiplies analysis cost across the volume without adding signal. GET /v1/simulation/template returns the value each template expects. example: true includeAutomaticMetrics: type: boolean default: true description: >- Let the run add metrics by itself off the attached flows, on top of the `metrics` named here. Two attach this way today: Agent Expectations wherever an attached flow has agent expectations written on it, and Keypad Entry wherever one has steps where the agent is expected to press keys. Both grade something authored on the flow that nothing else measures, which is why it is on by default. Set false when the `metrics` list is meant to be exhaustive: a plan testing only whether the caller can complete the flow may not want the agent graded on its expectations as well. False also pins the plan against any automatic metric Roark adds later. example: true enrichWithLiveConversation: type: boolean default: false description: >- Merge the customer's own recording of the real call into each simulation, so metrics can be scored against the live leg as well as the simulated one. This is the API equivalent of the dashboard's live-enrichment toggle. With this on, the run provisions a phone number and holds each call open for up to 15 minutes waiting for a matching call to be posted to POST /v1/call. A call matches on the provisioned number (`roarkPhoneNumber` on the job) with a start time inside the simulation window. If nothing arrives, the simulation still completes and any `LIVE`-sourced metric produces no value. Required by any metric whose `requiresLiveConversation` is true: without it that metric is silently skipped. example: false required: - direction - maxSimulationDurationSeconds - agentEndpoints - metrics additionalProperties: false description: 'The simulation to run: what to call, who calls it, and what to measure.' RunPlanFlowSelection: type: object properties: id: type: string format: uuid description: The customer flow to run. slug: type: string minLength: 1 description: >- The Roark-curated flow to run, by its stable slug. Use instead of `id` for a run you keep in version control: a curated flow’s id differs between deployments, its slug does not. Your own flows have no slug and are named by `id`. example: sf-prompt-injection happyPath: type: boolean description: >- Run the flow's happy path. Resolved when the run starts, so it follows the flow. edgeCases: anyOf: - type: string const: ALL - type: array items: $ref: '#/components/schemas/RunPlanEdgeCaseSelection' description: >- `"ALL"` runs every edge case the flow has when the run starts, so one added later is covered. An array runs only the ones you name, each able to carry its own persona override and values. personaOverrideId: type: - string - 'null' format: uuid description: >- Runs everything this attachment resolves as that persona instead of its own. variables: type: object additionalProperties: type: string description: Values for everything it resolves. example: customerName: John Doe appointmentDate: '2024-02-15' description: >- One customer flow attached to a run plan, and which of its ways of running you cover. Attaching the same flow more than once with different overrides is how you fan it out across personas or values. example: id: 550e8400-e29b-41d4-a716-446655440000 happyPath: true edgeCases: - id: 7c9e6679-7425-40de-944b-e07fc1f90ae7 variables: tier: premium RunPlanMetricRef: type: object properties: id: type: string format: uuid description: Metric definition UUID. Provide either this or `slug`, not both. slug: type: string minLength: 1 description: >- Stable metric slug (e.g. `customer_satisfaction`). Provide either this or `id`, not both. metricId: type: string minLength: 1 description: >- Alias of `slug` accepted for backwards compatibility. Use `slug` for new integrations. conversationSource: type: - string - 'null' enum: - SIMULATED - LIVE description: >- Which side of an enriched run this metric is scored on. Only meaningful with `enrichWithLiveConversation: true`, where a run has both a simulated conversation and the customer's own live recording of it. Defaults to `SIMULATED`. Use `LIVE` for a metric that must be measured against the real recording (audio quality, provider latency) rather than the simulated leg. `null` means the same as omitting it, so a plan read back from GET can be sent straight to PUT. additionalProperties: false RunPlanEdgeCaseSelection: type: object properties: id: type: string format: uuid description: The edge case to run. slug: type: string minLength: 1 description: >- The edge case to run, by its stable slug, matched within this flow. Use instead of `id` for a run you keep in version control: a curated edge case’s id differs between deployments and changes outright if it is renamed. Your own edge cases have no slug and are named by `id`. example: data-embedded-injection personaOverrideId: type: - string - 'null' format: uuid description: Run this one as that persona instead of its own. variables: type: object additionalProperties: type: string description: Values for this one only. example: customerName: John Doe appointmentDate: '2024-02-15' securitySchemes: Bearer: type: http scheme: bearer bearerFormat: JWT ````