> ## Documentation Index
> Fetch the complete documentation index at: https://docs.roark.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Overview
> Understand what metrics are and how to use them across monitoring and simulations
Metrics are how you measure what happens in every voice AI conversation. Roark collects some metrics automatically (like response time, talk time, and sentiment) and lets you define your own for things like compliance checks, task completion, or custom business KPIs.
**UI-first or code-first?** Author custom metrics and collectors in the dashboard, or define them as [config as code](/documentation/config-as-code/overview) - YAML in your git repo, applied with the [CLI](/documentation/sdks/cli). Both manage the same resources.
## Quickstart
Define a custom metric and run it on a call:
```typescript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
// Author an LLM-judge metric
const metric = await client.metric.createDefinition({
name: 'Resolved On First Call',
outputType: 'BOOLEAN',
llmPrompt: 'Did the agent resolve the issue without a transfer or callback?',
})
// Run it against an existing call, then read the result
await client.metricCollectionJob.create({
callIds: [''],
metrics: [{ id: metric.data.id }],
})
const results = await client.call.listMetrics('')
```
```python theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
metric = client.metric.create_definition(
name="Resolved On First Call",
output_type="BOOLEAN",
llm_prompt="Did the agent resolve the issue without a transfer or callback?",
)
client.metric_collection_job.create(
call_ids=[""],
metrics=[{"id": metric.data.id}],
)
results = client.call.list_metrics("")
```
```yaml roark/metrics/resolved-on-first-call.yaml theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
kind: metric
name: resolved_on_first_call
displayName: Resolved On First Call
type: BOOLEAN
prompt: |
Using {{transcript}} and {{world_context}}, did the agent resolve the
issue without a transfer or callback?
```
```bash theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
roark config apply ./roark
```
Attach it to a [collector](/documentation/metrics/metric-collectors) to score every matching call automatically.
Everything measurement-related lives in the **Measure** section of the sidebar:
* **Metrics**: your library of everything the project can measure, plus the collectors that run it on your calls
* **Studio**: where you author new metrics and run evaluation batteries against real calls
* **Datasets**: named collections of calls or chats for evaluation, comparison, and analysis
***
## What's a Metric?
A metric is a single measurement collected from a call. It has:
* **An output type**: boolean, numeric, scale, text, classification, or count
* **A scope**: global (one value per call) or per-participant (separate values for agent and customer)
* **A context**: call-level, segment-level (single utterance), or segment-range (span of conversation)
For example, `response_time` is a numeric, per-participant, segment-range metric that measures how long each speaker takes to respond. `identity_verified` might be a boolean, global, call-level metric powered by an LLM Judge prompt.
***
## Types of Metrics
### System Metrics (Built-in)
Roark automatically collects deterministic and voice-analysis metrics for every call. No configuration needed. Voice-analysis metrics are powered by Roark's custom voice analysis models, purpose-built to extract signal from conversational audio:
* **Performance**: Response time, talk time, silence duration, overlap/interruptions, latency
* **Emotion & Sentiment**: Sentiment tracking, 64+ emotion detection, vocal cues (raised voice, frustration), stress indicators
* **Speech**: Interruption detection, pause analysis, repetition detection
* **Compliance**: Disclosure completeness, prohibited language, PII handling, prompt injection resistance
* **Call Quality**: Speech quality scoring (DNSMOS), accent detection, voicemail handling
See the full list of 65+ system metrics in the [System Metrics Reference](/documentation/metrics/system-metrics). System metrics are read-only in Studio until you fork them with **Customize for your team**, which creates an org-scoped variant you can edit.
### Custom Metrics
Define your own metrics in [Studio](/documentation/metrics/studio) by picking one of three engines:
| Engine | What it does | Example |
| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
| **LLM Judge** | Scores each call against a natural-language prompt, evaluated by **Roark Prism**, our purpose-built evaluation model for voice AI conversations | "Did the agent verify the caller's identity?" → Boolean |
| **Pattern** | Composes triggers, outcomes, and time windows to detect when something happens in a call | Fire when the agent mentions a refund within 30s of a complaint |
| **Formula** | Computes a value from other metrics into a composite score | Weighted quality score across empathy, resolution, and latency |
LLM Judge metrics return typed results:
* "Did the agent verify the caller's identity?" → **Boolean**
* "Rate the agent's empathy on a scale of 1-10" → **Scale**
* "What was the primary reason for the call?" → **Classification**
* "How many times did the agent attempt to upsell?" → **Count**
Custom metrics are created in Studio's Author mode (**New metric** on the Metrics page) or via the [SDK](/documentation/metrics/custom-metrics#create-a-metric-definition). See [Custom Metrics](/documentation/metrics/custom-metrics) for the full authoring guide.
***
## The Metrics Page
The Metrics page (**Measure → Metrics**) is home base for measurement:
* **Collectors strip**: each collector runs a set of metrics on the calls its segment matches. Compact cards show each collector's name, segment, and metrics, with a **Manage collectors** link and a **New collector** tile.
* **Library**: everything this project can measure, grouped by package. Filter with the **All / System / Custom** pills or search; clicking a metric opens it in Studio for editing and testing.
* **New metric**: jumps straight into Studio's Author mode.
***
## How It All Fits Together
Here's the typical sequence from defining a metric to seeing results:
Use a built-in [system metric](/documentation/metrics/system-metrics), or [create your own custom metric](/documentation/metrics/custom-metrics) with an LLM Judge prompt, a Pattern, or a Formula.
Author mode's test rail lets you [run your metric against real calls](/documentation/metrics/studio): add test calls and hit **Run all** to validate it produces the results you expect. Iterate on the prompt until you're satisfied. To compare many metrics across many calls at once, use Evaluate mode's **Run battery**.
Set [pass/fail criteria](/documentation/metrics/thresholds) on your metric, for example, `Customer Satisfaction >= 7` or `Response Time < 1000ms`. Thresholds turn raw values into actionable outcomes.
Choose how and when your metric runs:
* **[Collectors](/documentation/metrics/metric-collectors)**: Automate collection on incoming calls (monitoring). Add conditions to target specific agents, sources, or call properties; leave them empty to match every call.
* **[Simulation plans](/documentation/simulation-testing/run-plans)**: Attach metrics with thresholds to simulation runs to validate agent behavior before deployment.
* **[Collection Jobs](/documentation/metrics/metric-collection-jobs)**: Run metrics on demand against existing calls via the SDK, useful for backfilling or re-processing.
View metric values per call in [Call History](/documentation/observability/live-monitoring), aggregate them in [Reports](/documentation/observability/reports), and organize everything in [Dashboards](/documentation/observability/dashboards). Group the calls you care about into [Datasets](/documentation/metrics/datasets) for evaluation and comparison.
***
## Quick Start Examples
The REST API and SDKs keep the older name for collectors: the SDK client is `client.metricPolicy.*` and the endpoints live under `/v1/metric/policies`. In the UI, these are **Collectors**.
System metrics like `response_time` are already collected for every call. To set a quality bar, create a collector with a threshold:
```typescript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
const collector = await client.metricPolicy.create({
name: 'Agent Response Time SLA',
status: 'ACTIVE',
conditions: [
{
conditions: [
{
conditionType: 'AGENT',
conditionKey: 'your-agent-id',
},
],
},
],
metrics: [
{
id: 'response-time-metric-id',
threshold: {
operator: 'LESS_THAN',
value: 1000,
aggregationMode: 'P95',
participantRole: 'AGENT',
},
},
],
})
```
Calls where the agent's P95 response time exceeds 1 second are automatically flagged as failures.
Define a business-specific metric and add it to a collector. Roark Prism evaluates each call against your prompt:
```typescript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
// 1. Create the metric definition
const metric = await client.metric.createDefinition({
name: 'Appointment Booked',
outputType: 'BOOLEAN',
analysisPackageId: 'your-package-id',
llmPrompt: 'Did the agent successfully book an appointment for the caller?',
booleanTrueLabel: 'Booked',
booleanFalseLabel: 'Not Booked',
})
// 2. Add it to a collector so it runs on every call
const collector = await client.metricPolicy.create({
name: 'Appointment Booking Check',
status: 'ACTIVE',
metrics: [{ id: metric.data.id }],
})
```
Test your prompt in [Studio](/documentation/metrics/studio) first to validate it produces the results you expect.
Attach metrics with thresholds to a simulation plan to validate agent behavior before deployment. Run the simulation, and each call is scored against your pass/fail criteria:
```typescript theme={"theme":{"light":"everforest-light","dark":"everforest-dark"}}
// Run metrics against simulation calls after they complete
const job = await client.metricCollectionJob.create({
callIds: ['sim-call-1', 'sim-call-2', 'sim-call-3'],
metrics: [
{ id: 'task-completion-metric-id' },
{ id: 'compliance-check-metric-id' },
],
})
// Check results
const result = await client.metricCollectionJob.getByID(job.data.id)
console.log(`${result.data.completedItems}/${result.data.totalItems} processed`)
```
Or configure metrics with [thresholds](/documentation/metrics/thresholds) directly in a [simulation plan](/documentation/simulation-testing/run-plans) from the dashboard.
***
## Sections
Browse all 65+ built-in metrics powered by specialized models
Create custom metrics with LLM Judge prompts, patterns, and formulas
Author metrics and run evaluation batteries against real calls
Run sets of metrics on the calls each collector's segment matches
Define pass/fail criteria for your metrics
Group calls or chats for evaluation, comparison, and analysis
Run metrics on demand for existing calls via the SDK