Alerts
Define and manage alerts. Alerts evaluate event metrics on a schedule and trigger notifications when configured thresholds are crossed. (3 commands)
Charts
Define and manage saved charts. Charts are visualizations that aggregate metrics over time with bucketing, filters, and groupings. (5 commands)
Datapoints
Manage individual records inside datasets, including batch creation and mapping to source events. (6 commands)
Datasets
Curate collections of datapoints used as test sets for evaluations and experiments. (6 commands)
Events
Read and write trace events. Events are the spans that capture every step of an AI application’s execution. (5 commands)
Experiments
Run, retrieve, and compare evaluation runs to measure how prompt or configuration changes affect agent performance. (11 commands)
Metric Versions
Snapshot, list, and deploy versions of a metric’s definition so changes can be reviewed and rolled back without losing history. (3 commands)
Metrics
Define and run evaluators, i.e. automated quality checks that score traces against criteria like accuracy, safety, or correctness. (5 commands)
Projects
Create and manage projects within a workspace. A project is the container for the events, datasets, evaluations, and alerts logged against it. (4 commands)
Sessions
Group related trace events into sessions, the top-level container for a multi-step or multi-service AI interaction. (2 commands)
Virtual Dataplanes
Create and manage virtual data planes. A virtual data plane is a logical tenant boundary inside an organization, hosted on a physical cluster; several virtual data planes commonly share one cluster. Workspaces live inside a virtual data plane. (4 commands)
Workspaces
Read and manage workspaces. A workspace groups the projects belonging to one team or environment, and owns the API keys and AI provider secrets its projects share. (4 commands)