> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeyhive.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Comparing Experiments

> Learn how to compare multiple experiments in HoneyHive to spot improvements and regressions.

Comparison view allows you to run multiple experiments using the same dataset (linked by `dataset_id`) and compare their results side-by-side. The dataset is the input data for the experiment which can be through HoneyHive Datsets or inputs passed. This is particularly useful when you want to benchmark different models, prompts, or configurations against each other.

<Frame>
  <img src="https://mintcdn.com/honeyhiveai/qmpHooEVX6j-ieIE/images/comparitive_evals_screenshot.png?fit=max&auto=format&n=qmpHooEVX6j-ieIE&q=85&s=5d670083096f94d4380834c60c01fd60" width="2622" height="1516" data-path="images/comparitive_evals_screenshot.png" />
</Frame>

The comparison view allows you to:

* [step level comparisons](#1-step-level-comparisons)
* [view metric aggregates](#2-aggregated-metrics)
* [find improved/regressed events](#3-improved-regressed-events)
* [compare outputs](#4-output-diff-viewer)
* [analyze metric distribution](#5-metric-distribution)

## Navigating the Comparison View

Let's walk through the key features of the comparison view to help you effectively compare your experiments.

<div style={{ position: 'relative', boxSizing: 'content-box', maxHeight: '80vh', maxHeight: '80svh', width: '100%', aspectRatio: '1.8421052631578947', padding: '40px 0 40px 0' }}>
  <iframe src="https://app.supademo.com/embed/cm3j0uc540c1182qcfhwp164q?embed_v=2" loading="lazy" title="Comparison View" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style={{ position: 'absolute', top: '0', left: '0', width: '100%', height: '100%' }} />
</div>

## Advanced Comparison Features

### 1. Step Level Comparisons

HoneyHive allows you to compare experiments at each individual step level, giving you granular insights into how different configurations perform at specific stages of your workflow.

<div style={{ position: 'relative', boxSizing: 'content-box', maxHeight: '80vh', maxHeight: '80svh', width: '100%', aspectRatio: '1.8421052631578947', padding: '40px 0 40px 0' }}>
  <iframe src="https://app.supademo.com/embed/cm3j3pzev0cyq82qcgk1kwbv3?embed_v=2" loading="lazy" title="Step Level Comparison" allow="clipboard-write" frameborder="0" webkitallowfullscreen="true" mozallowfullscreen="true" allowfullscreen style={{ position: 'absolute', top: '0', left: '0', width: '100%', height: '100%' }} />
</div>

### 2. Aggregated Metrics

HoneyHive automatically calculates and compares aggregates from:

* Server-side metrics
* Client-side metrics
* Composite metrics at the session level

<Frame>
  <img src="https://mintcdn.com/honeyhiveai/qmpHooEVX6j-ieIE/images/comparitive_evals_aggregates.png?fit=max&auto=format&n=qmpHooEVX6j-ieIE&q=85&s=6d2fc96c08a300fa34c261c467c226b1" width="858" height="714" data-path="images/comparitive_evals_aggregates.png" />
</Frame>

### 3. Improved/regressed events

Filter for events that have improved or regressed in specific metrics.

Select the metric and operation you want.

<Frame>
  <img src="https://mintcdn.com/honeyhiveai/qmpHooEVX6j-ieIE/images/comparitive_evals_filter_select.png?fit=max&auto=format&n=qmpHooEVX6j-ieIE&q=85&s=c87709b795311a851a717318a5f130e3" width="850" height="690" data-path="images/comparitive_evals_filter_select.png" />
</Frame>

View the corresponding events in the events table.

<Frame>
  <img src="https://mintcdn.com/honeyhiveai/qmpHooEVX6j-ieIE/images/comparitive_evals_filter_view.png?fit=max&auto=format&n=qmpHooEVX6j-ieIE&q=85&s=c0d09259296a1b0e7a20aced2d6864ee" width="2586" height="930" data-path="images/comparitive_evals_filter_view.png" />
</Frame>

### 4. Output Diff Viewer

Compare outputs and metrics of corresponding events with the same event name.

<Frame>
  <img src="https://mintcdn.com/honeyhiveai/qmpHooEVX6j-ieIE/images/comparitive_evals_diff.png?fit=max&auto=format&n=qmpHooEVX6j-ieIE&q=85&s=e01194418e075d994b078228fd7dbd20" width="2554" height="1116" data-path="images/comparitive_evals_diff.png" />
</Frame>

### 5. Metric Distribution

Analyze the distribution of various metrics for deeper insights.

<Frame>
  <img src="https://mintcdn.com/honeyhiveai/qmpHooEVX6j-ieIE/images/comparitive_evals_distribution.png?fit=max&auto=format&n=qmpHooEVX6j-ieIE&q=85&s=68c343008fa51fb0c1ca6de800858caf" width="1716" height="686" data-path="images/comparitive_evals_distribution.png" />
</Frame>

## Best Practices

1. Use a consistent dataset for all compared experiments.
2. Isolate one change at a time (e.g., model, prompt, temperature) to understand its specific impact.
3. Ensure a sufficient sample size for statistically significant conclusions.
4. Document configurations used in each experiment for future reference.

## Conclusion

Comparitive View for Experiments in HoneyHive provide a powerful tool for benchmarking different LLM configurations. Leverage this feature to make data-driven decisions about optimal models, prompts, or parameters for your specific use case.