> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeyhive.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Experiments

> Use the HoneyHive CLI to run, retrieve, and compare evaluation runs. Measure how prompt or configuration changes affect agent performance on datasets.

## Schema introspection

Every command below with arguments supports two read-only flags for tooling and AI agents:

* `--show-file-schema`: print the JSON Schema for the full request object (the format `--filename` accepts).
* `--show-argument-schema <flag-name>`: print the JSON Schema for one argument's value. Pass the kebab flag name **without** the leading `--` (e.g. `dataset-id`, not `--dataset-id`).

Both write pure JSON to stdout and never call the API. They cannot be combined with any other command-specific flag.

## `list-runs`

Get a list of evaluation runs

List experiment runs with optional filtering by dataset, status, name, date range, and specific run IDs. Results are paginated and sortable.

### Usage

```sh theme={null}
honeyhive experiments list-runs [options]
```

### Options

| Flag           | Type   | Required | Description                                                                             |
| -------------- | ------ | -------- | --------------------------------------------------------------------------------------- |
| `--dataset-id` | string | no       | Filter by dataset ID                                                                    |
| `--date-range` | json   | no       | Filter by date range                                                                    |
| `--limit`      | number | no       | Number of results per page                                                              |
| `--name`       | string | no       | Filter by run name                                                                      |
| `--page`       | number | no       | Page number for pagination                                                              |
| `--run-ids`    | json   | no       | List of specific run IDs to fetch                                                       |
| `--sort-by`    | string | no       | Field to sort by Allowed: `created_at`, `updated_at`, `name`, `status`.                 |
| `--sort-order` | string | no       | Sort order Allowed: `asc`, `desc`.                                                      |
| `--status`     | string | no       | Filter by run status Allowed: `pending`, `completed`, `failed`, `cancelled`, `running`. |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `create-run`

Create a new evaluation run

Create a new experiment run to track an evaluation against a dataset.

### Usage

```sh theme={null}
honeyhive experiments create-run [options]
```

### Options

| Flag               | Type   | Required | Description                                                               |
| ------------------ | ------ | -------- | ------------------------------------------------------------------------- |
| `--configuration`  | json   | no       | configuration                                                             |
| `--datapoint-ids`  | json   | no       | datapoint\_ids                                                            |
| `--dataset-id`     | string | no       | dataset\_id                                                               |
| `--description`    | string | no       | description                                                               |
| `--evaluators`     | json   | no       | evaluators                                                                |
| `--event-ids`      | json   | no       | event\_ids                                                                |
| `--metadata`       | json   | no       | metadata                                                                  |
| `--name`           | string | no       | name                                                                      |
| `--passing-ranges` | json   | no       | passing\_ranges                                                           |
| `--results`        | json   | no       | results                                                                   |
| `--run-id`         | string | no       | run\_id                                                                   |
| `--session-ids`    | json   | no       | session\_ids                                                              |
| `--status`         | string | no       | status Allowed: `pending`, `completed`, `failed`, `cancelled`, `running`. |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `get-runs-schema`

Get events schema across all experiment runs in a project

Retrieve the aggregated events schema (fields, datasets, mappings) across all experiment runs in the project.

### Usage

```sh theme={null}
honeyhive experiments get-runs-schema [options]
```

### Options

| Flag           | Type | Required | Description          |
| -------------- | ---- | -------- | -------------------- |
| `--date-range` | json | no       | Filter by date range |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `get-run`

Get details of an evaluation run

Retrieve the full details of a single experiment run by its run ID.

### Usage

```sh theme={null}
honeyhive experiments get-run [options]
```

### Options

| Flag       | Type   | Required | Description |
| ---------- | ------ | -------- | ----------- |
| `--run-id` | string | yes      | run\_id     |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `update-run`

Update an evaluation run

Update fields on an existing experiment run such as name, status, metadata, or results.

### Usage

```sh theme={null}
honeyhive experiments update-run [options]
```

### Options

| Flag               | Type   | Required | Description                                                               |
| ------------------ | ------ | -------- | ------------------------------------------------------------------------- |
| `--run-id`         | string | yes      | run\_id                                                                   |
| `--configuration`  | json   | no       | configuration                                                             |
| `--datapoint-ids`  | json   | no       | datapoint\_ids                                                            |
| `--description`    | string | no       | description                                                               |
| `--evaluators`     | json   | no       | evaluators                                                                |
| `--event-ids`      | json   | no       | event\_ids                                                                |
| `--metadata`       | json   | no       | metadata                                                                  |
| `--name`           | string | no       | name                                                                      |
| `--passing-ranges` | json   | no       | passing\_ranges                                                           |
| `--results`        | json   | no       | results                                                                   |
| `--session-ids`    | json   | no       | session\_ids                                                              |
| `--status`         | string | no       | status Allowed: `pending`, `completed`, `failed`, `cancelled`, `running`. |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `delete-run`

Delete an evaluation run

Permanently delete an experiment run by its run ID.

### Usage

```sh theme={null}
honeyhive experiments delete-run [options]
```

### Options

| Flag       | Type   | Required | Description |
| ---------- | ------ | -------- | ----------- |
| `--run-id` | string | yes      | run\_id     |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `get-run-schema`

Get events schema for a single experiment run

Retrieve the events schema (fields, datasets, mappings) for a single experiment run.

### Usage

```sh theme={null}
honeyhive experiments get-run-schema [options]
```

### Options

| Flag           | Type   | Required | Description                |
| -------------- | ------ | -------- | -------------------------- |
| `--run-id`     | string | yes      | Experiment run ID (UUIDv4) |
| `--date-range` | json   | no       | Filter by date range       |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `get-run-metrics`

Get event metrics for an experiment run

Retrieve event metrics from ClickHouse for a specific experiment run

### Usage

```sh theme={null}
honeyhive experiments get-run-metrics [options]
```

### Options

| Flag           | Type   | Required | Description                                                        |
| -------------- | ------ | -------- | ------------------------------------------------------------------ |
| `--run-id`     | string | yes      | Experiment run ID (UUIDv4)                                         |
| `--date-range` | string | no       | Date range filter as JSON string                                   |
| `--filters`    | json   | no       | Optional filters to apply (JSON string or array of filter objects) |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `get-summary`

Retrieve experiment summary

Compute evaluation summary for an experiment run: pass/fail results, metric aggregations, per-datapoint results, event details, and the experiment run object.

### Usage

```sh theme={null}
honeyhive experiments get-summary [options]
```

### Options

| Flag                   | Type   | Required | Description                                                                                                               |
| ---------------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------- |
| `--run-id`             | string | yes      | Experiment run ID (UUIDv4)                                                                                                |
| `--aggregate-function` | string | no       | Aggregation function to apply to metrics Allowed: `average`, `min`, `max`, `median`, `p95`, `p99`, `p90`, `sum`, `count`. |
| `--filters`            | json   | no       | Optional filters to apply (JSON string or array of filter objects)                                                        |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `compare-runs`

Retrieve experiment comparison

Compare metrics and results between two experiment runs

### Usage

```sh theme={null}
honeyhive experiments compare-runs [options]
```

### Options

| Flag                   | Type   | Required | Description                                                                                                               |
| ---------------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------- |
| `--new-run-id`         | string | yes      | New experiment run ID to compare (UUIDv4)                                                                                 |
| `--old-run-id`         | string | yes      | Old experiment run ID to compare against (UUIDv4)                                                                         |
| `--aggregate-function` | string | no       | Aggregation function to apply to metrics Allowed: `average`, `min`, `max`, `median`, `p95`, `p99`, `p90`, `sum`, `count`. |
| `--filters`            | json   | no       | Optional filters to apply (JSON string or array of filter objects)                                                        |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.

## `compare-run-events`

Compare events between two experiment runs

Retrieve and compare events between two experiment runs for detailed analysis

### Usage

```sh theme={null}
honeyhive experiments compare-run-events [options]
```

### Options

| Flag           | Type   | Required | Description                                        |
| -------------- | ------ | -------- | -------------------------------------------------- |
| `--new-run-id` | string | yes      | New experiment run ID (UUIDv4)                     |
| `--old-run-id` | string | yes      | Old experiment run ID to compare against (UUIDv4)  |
| `--event-name` | string | no       | Filter by event name                               |
| `--event-type` | string | no       | Filter by event type                               |
| `--filter`     | json   | no       | Additional filter criteria (JSON string or object) |
| `--limit`      | number | no       | Maximum number of results                          |
| `--page`       | number | no       | Page number for pagination                         |

Also supports `--show-file-schema`, `--show-argument-schema <flag-name>`, and `--filename`. See [Schema introspection](#schema-introspection) for details.