Skip to main content
You do not need server-side evaluators to run experiments in HoneyHive. If you already score your AI app offline - in a notebook, a CI job, or your own evaluation harness - you can sync those results into an experiment run and get the same dashboard, comparison, and regression-detection views. Your scores can come from any framework, or none.

​
When to use this

  • Score interactions that already live in a spreadsheet, CSV, or database table
  • Benchmark past prompts or model versions with new metrics, without rerunning the original generation
  • Sync results from an evaluation harness or framework you already use
  • Run experiments from a language or runtime where the HoneyHive SDK isn’t available

​
How an experiment is stored

An experiment run is a set of traced sessions grouped by a shared run_id. Each session carries:
FieldWhat it holds
inputsThe test case
outputsWhat your application produced
metricsThe scores you computed (numbers, booleans, or strings)
Where those metrics come from - a HoneyHive evaluator, a third-party library, or a hand-written function - makes no difference to the dashboard.

​
Choose your path

Two paths, same three steps (create a run, log each result with your metrics, complete the run):
  • Python SDK - typed HoneyHive client
  • REST API - plain HTTP from any language, no HoneyHive packages

​
Sync via the Python SDK

The HoneyHive client wraps the three calls with typed methods: create a run, log each result as a session carrying your precomputed metrics, then complete the run. 
import uuid
from honeyhive import HoneyHive
from honeyhive.models import PostExperimentRunRequest, PutExperimentRunRequest

client = HoneyHive()  # reads HH_API_KEY (and HH_API_URL) from your environment

# Results you computed offline, with any framework (or none).
results = [
    {
        "inputs": {"question": "What is the capital of France?"},
        "outputs": {"answer": "Paris"},
        "metrics": {"exact_match": 1.0, "relevance": 0.95},
    },
    {
        "inputs": {"question": "Who wrote Hamlet?"},
        "outputs": {"answer": "Shakespeare"},
        "metrics": {"exact_match": 1.0, "relevance": 0.88},
    },
]

# 1. Create a pending experiment run (the project is scoped by your API key)
run = client.experiments.create_run(
    PostExperimentRunRequest(name="offline-eval-v1", event_ids=[], status="pending")
)
run_id = run.run_id

# 2. Log each result as a session, attaching your precomputed scores as metrics
session_ids = []
for item in results:
    response = client.sessions.start(
        {
            "session": {
                "session_name": "offline-eval-v1",
                "source": "evaluation",
                "session_id": str(uuid.uuid4()),
                "inputs": item["inputs"],
                "outputs": item["outputs"],
                "metrics": item["metrics"],
                "metadata": {"run_id": run_id, "datapoint_id": str(uuid.uuid4())},
            }
        }
    )
    session_ids.append(response.session_id)

# 3. Complete the run, linking every session
client.experiments.update_run(
    run_id, PutExperimentRunRequest(event_ids=session_ids, status="completed")
)
metadata.run_id on each session is what links it to the run and rolls its metrics up. Set metadata.datapoint_id too, so cross-run comparison can pair the same test case across runs.
Logs in a CSV or database? With pandas, df.to_dict("records") turns each row into a dict - map your columns into inputs, outputs, and metrics, then loop over the list.
Want HoneyHive to run your scoring during the experiment instead of syncing precomputed metrics? Use client-side evaluators with evaluate().

​
Sync via the REST API (no packages)

No HoneyHive packages installed? The same three calls work as plain HTTP requests, from any language or a cURL script. 
import os
import uuid
import requests

HH_API_KEY = os.environ["HH_API_KEY"]
# Standard cloud uses https://api.dp1.us.honeyhive.ai. Self-hosted/dedicated
# deployments set their own host (the value of HH_API_URL).
BASE_URL = os.getenv("HH_API_URL", "https://api.dp1.us.honeyhive.ai")
headers = {"Authorization": f"Bearer {HH_API_KEY}", "Content-Type": "application/json"}

# Results you computed offline, with any framework (or none).
results = [
    {
        "inputs": {"question": "What is the capital of France?"},
        "outputs": {"answer": "Paris"},
        "metrics": {"exact_match": 1.0, "relevance": 0.95},
    },
    {
        "inputs": {"question": "Who wrote Hamlet?"},
        "outputs": {"answer": "Shakespeare"},
        "metrics": {"exact_match": 1.0, "relevance": 0.88},
    },
]

# 1. Create a pending experiment run (the project is scoped by your API key)
run = requests.post(
    f"{BASE_URL}/v1/runs",
    headers=headers,
    json={"name": "offline-eval-v1", "event_ids": [], "status": "pending"},
).json()
run_id = run["run_id"]

# 2. Log each result as a session, attaching your precomputed scores as metrics
session_ids = []
for item in results:
    response = requests.post(
        f"{BASE_URL}/session/start",
        headers=headers,
        json={
            "session": {
                "session_name": "offline-eval-v1",
                "source": "evaluation",
                "session_id": str(uuid.uuid4()),
                "inputs": item["inputs"],
                "outputs": item["outputs"],
                "metrics": item["metrics"],
                "metadata": {
                    "run_id": run_id,
                    "datapoint_id": str(uuid.uuid4()),
                },
            }
        },
    )
    session_ids.append(response.json()["session_id"])

# 3. Complete the run, linking every session
requests.put(
    f"{BASE_URL}/v1/runs/{run_id}",
    headers=headers,
    json={"event_ids": session_ids, "status": "completed"},
)
For the full endpoint reference, including how to attach a managed dataset or log multi-step traces alongside your scores, see Experiments via API.

​
Viewing results

Open Experiments in the dashboard to see your synced run. Each session appears as a row with its inputs, outputs, and metric columns, and you can compare runs to spot improvements and regressions across versions.

​
Next steps

Experiments via API

Full REST endpoint reference for runs, sessions, and events

Client-Side Evaluators

Write scoring functions that run in your environment

Compare Experiments

Identify improvements and regressions across runs

Custom Metrics

Attach scores and guardrail results to any trace