Custom Charts

Quick Start: Building Your First Chart

Creating insightful visualizations in HoneyHive is straightforward.

Access Discover

Click New Chart in your Dashboard, or navigate to the Discover tab from the sidebar.

Select Your Data Source

Choose from three data scopes:

Scope	What it covers
Sessions	Full user interactions/traces (entire conversations)
Completions	Individual LLM calls
All Events	Any tracked step in your pipeline, including tool calls

Configure Your Visualization

Setting	Description
Event	Which event type to analyze (default: All Sessions/Completions/Events)
Metric	What to measure (e.g., Request Volume, Duration, Cost, or custom evaluators)
Aggregation	How to calculate (Sum, Average, Median, 99th Percentile, etc.)

Refine Your Analysis (Optional)

Setting	Description
Filter	Narrow down to specific data segments (e.g., `source = "production"`)
Group By	Split results by properties (e.g., `prompt_version`, `model`, `user_tier`)
Time Range	Set your analysis window (1d, 7d, 30d, etc.)

Understanding Your Data

To build effective charts, it helps to understand the data components available in HoneyHive.

Metrics

Metrics are the numerical values you visualize in charts.

Usage Metrics

Metric	What it tells you
Request Volume	Queries over time. Spot usage spikes or drops.
Cost	Direct expenses. See if that new feature is breaking the bank.
Duration	System latency. Slow responses kill engagement.

Evaluators

Your custom quality checks, either Python or LLM-based. Must return float or boolean to chart.

Example	Type	Question it answers
Keyword Presence	`boolean`	Does every product review mention the product?
Coherence Score	`float`	How logically sound are multi-turn conversations?

User Feedback

The voice of your users, quantified. Accepts float or boolean inputs.

Example	Type	Question it answers
Usefulness Rating	`float`	On a scale of 1-5, how useful was this response?
Used in Report	`boolean`	Did the user actually use this in their report?

Properties

Properties provide context for your metrics. All properties in the enrichment schema such as config, user properties, feedback, metrics, and metadata can be used to slice and dice your data.

Metrics chart performance. Properties unveil the context behind that performance. Both are crucial for exploratory data analysis.

Chart Types

Each chart type focuses on a different part of your LLM pipeline.

Completions
Sessions
Events

Focus: Individual LLM calls.Key Metrics: cost, duration, tokens, errors, and any specified evaluators.

Example use case

Hypothesis: “Longer user messages cause more token waste.”Test: Chart Average Unused Output Tokens grouped by binned_input_length.

Focus: Full multi-turn user interactions and entire traces.Key Metrics: User Turns, Session Duration, Avg User Rating, Task Completion Rate.

Example use case

Hypothesis: “Agents start looping after n turns.”Test: Chart Agent Trajectory Evaluator grouped by Number of turns.

Focus: Specific agents, tools, or steps in your pipeline.Key Metrics: Retrieval Latency, Synthesis Quality, Tool Choice Accuracy.

Example use case

Hypothesis: “Our reranker is the bottleneck in high-load scenarios.”Test: Chart 99th Percentile Rerank Time vs. Requests per Minute.

Programmatic Chart Management

Use the first-party TypeScript API SDK or HoneyHive CLI to create, list, retrieve, update, and delete dashboard charts from scripts or CI jobs. In the TypeScript API SDK, use the client.charts namespace. In the CLI, use the honeyhive charts commands. Project API keys need the corresponding project.chart.* permissions.

honeyhive charts list

Getting Started

Observability

Evaluation

Prompt Management

Administration

Learn More

Quick Start: Building Your First Chart

Understanding Your Data

Metrics

Properties

Chart Types

Programmatic Chart Management

Next Steps

Set Up Alerts

Online Evaluations

​Quick Start: Building Your First Chart

​Understanding Your Data

​Metrics

​Properties

​Chart Types

​Programmatic Chart Management

​Next Steps

Set Up Alerts

Online Evaluations

Quick Start: Building Your First Chart

Understanding Your Data

Metrics

Properties

Chart Types

Programmatic Chart Management

Next Steps