Skip to main content
Alert creation interface
This guide walks you through creating an alert. For an overview of alert types, states, and management actions, see Alerts Overview.

​
Create an Alert

1

Name and describe your alert

Give your alert a clear name and description so your team knows what it monitors.
2

Choose monitoring frequency

Select how often to evaluate your alert:
  • Hourly: Checked every hour for immediate detection
  • Daily: Checked every hour but evaluated over daily periods
  • Weekly: Checked daily but evaluated over weekly periods
  • Monthly: Checked daily but evaluated over monthly periods
Note: More frequent checking means faster detection, even for longer evaluation periods.
3

Select what to monitor

Event Type: Choose the data source for your alert
  • Models: Individual LLM API calls and completions
  • Sessions: Complete user conversations or workflows
  • Tools: Function calls and external integrations
  • Chains: Multi-step workflows and complex pipelines
Event Filters: Narrow down to specific segments (optional)
  • Filter by event name, tenant, or any custom metadata
  • Example: Only monitor production traffic or specific model versions
4

Define the metric and aggregation

Metric: What to measure. The available metrics depend on the event type you selected:For Models, Tools, and Chains:
  • duration - Span latency
  • metadata.* - Any metadata field (e.g., metadata.total_tokens, metadata.cost)
  • metrics.* - Any custom metric or evaluator score (e.g., metrics.faithfulness, metrics.relevance)
  • event_id - Event count (use with COUNT aggregation)
For Sessions, you can alert on aggregate fields computed across all child spans in the session:
  • duration - Total session wall-clock duration
  • metadata.cost - Total cost across all LLM calls in the session
  • metadata.total_tokens - Total tokens (prompt + completion) across the session
  • metadata.prompt_tokens - Total prompt tokens across the session
  • metadata.completion_tokens - Total completion tokens across the session
  • metadata.num_events - Number of child spans in the session
  • metadata.num_model_events - Number of LLM call spans in the session
  • metadata.has_feedback - Boolean indicating whether the session has received user feedback
Aggregation: How to summarize the data
  • COUNT: Total number of events
  • AVERAGE: Mean value across events
  • P90/P95/P99: Percentile values for performance monitoring
  • SUM: Total of all values
  • MIN/MAX: Extreme values
  • MEDIAN: Middle value
5

Set alert thresholds

Alert Type: Choose between aggregate alerts (absolute thresholds) or drift alerts (relative change detection)Trigger Threshold: The value that triggers the alert
  • Aggregate example: “Average latency > 2 seconds”
  • Drift example: “25% worse than previous period”
Resolution Threshold: When the alert auto-resolves (defaults to the inverse of your trigger threshold)
6

Configure notifications

  • Channel: Email is available by default. Slack (via incoming webhook) and Webhook (HTTP endpoint) channels are also supported - self-hosted customers can enable them via environment variables, and cloud customers can contact us to get them enabled.
  • Recipients: Choose specific team members or notify all project members
Notifications include the metric value that triggered the alert and a direct link to investigate.

​
Next Steps

Once your alert is created, you can manage it from the alerts page. See Alerts Overview for details on alert states and actions like pausing, muting, and resolving alerts.