HoneyHive Docs

Python
TypeScript

While the HoneyHive tracer automatically instruments all of the most common LLM application components, you might sometimes want to trace parts of your code beyond the automated coverage.To do this, you can use HoneyHive’s manual tracing tools, which we’ll walk through in this guide.

Prerequisites

Please complete the Quickstart and make sure you have initialized your tracer using HoneyHiveTracer.init().

Overview

HoneyHive gives you full customizability over your traces.You can trace any function in your code and see its inputs, outputs, errors, duration, etc. by decorating it with the @trace decorator.

The @atrace decorator is identical in functionality to the @trace decorator, but is used for async functions.

from honeyhive import trace

@trace
def my_function(param1, param2):
    # Code here
    return result

Enriching Custom Spans

You can enrich your custom spans with additional properties to provide more context to your traces. There are 2 ways to enrich spans in HoneyHive:-

Enriching before execution: The decorator-based enrichment below shows how you can enrich static properties before the code executes.
Enriching during execution: You can enrich dynamic properties on a session/trace or event/span level using enrich_session and enrich_span functions. Please refer to the Enrich Traces documentation for more details on enrichment during execution.

Enriching Before Execution Using Decorators

Here is an example of how to enrich the Configuration, Metadata, and Event Type fields on the decorator:

Python

from honeyhive import trace

@trace(
    config={
        "event_type": "my_event_type",
        "metadata": {"key": "value"},
    },
    metadata={
        "key": "value",
    },
    event_type="chain",
)
def my_function(param1, param2):
    # Code here
    return result

Enriching During Execution Using `enrich_span` or `enrich_session`

You can also enrich your traces and custom spans with various propertires during execution using the enrich_span or enrich_session functions.Please refer to the Enrich Traces documentation for more details on enrichment during execution. You can find the complete documentation for this in the Python SDK reference.

Async Function Tracing with @atrace

For async functions in Python, use the @atrace decorator instead of @trace. The @atrace decorator is specifically designed to handle async functions properly:

from honeyhive import atrace

@atrace
async def my_async_function(param1: str) -> str:
    # Async operations here
    result = await some_async_operation(param1)
    return result

# The function can be called with await as normal
result = await my_async_function("test")

The @atrace decorator supports the same enrichment options as @trace:

@atrace(
    config={"event_type": "my_event_type"},
    metadata={"key": "value"},
    event_type="chain"
)
async def my_async_function(param1: str) -> str:
    result = await some_async_operation(param1)
    return result

Best practices

Use descriptive function names, as they will be used as span names in your traces
Be mindful of sensitive data when tracing functions - avoid capturing passwords or other secrets

By strategically adding the @trace decorator to key functions in your application, you can gain valuable insights into your custom code’s performance and behavior, complementing HoneyHive’s automatic instrumentation of LLM components.

Visualizing Your Traces

All traces can be visualized in the HoneyHive web app under Log Store in the session’s specified project. The session tree view shows you the execution flow of your trace, as well all the automatically captured and manually enriched attributes.

Next Up: Enrich Your Traces

Here are some ways in which you can enrich your traces:

Enrichments Overview

An overview of the types of enrichments available for your traces and spans

Logging Client-Side Evaluations

Add client-side metrics, guardrail results, or evaluations to your traces

Logging User Feedback

Enrich traces with feedback or additional annotations

Logging Configuration Details

Include config details, model hyperparameters, or prompt templates

Logging User Properties

Add user properties or tracking IDs to your traces

Logging Custom Metadata

Enrich traces with arbitrary metadata or JSON

While HoneyHive automatically instruments many common LLM application components, you may sometimes want to trace specific parts of your code beyond the automated coverage. HoneyHive provides flexible tracing methods including the general-purpose traceFunction as well as event-specific methods (traceModel, traceTool, traceChain) that allow you to easily add custom spans to your TypeScript application. These methods capture function inputs, outputs, and additional metadata, providing deeper insights into your application’s behavior.

Unlike our auto-tracer, this approach works with ESModules automatically.

Using the `traceFunction` method

To use the traceFunction method, you first need to initialize the HoneyHiveTracer:

import { HoneyHiveTracer } from "honeyhive";

const HH_API_KEY: string = process.env.HH_API_KEY || "";
const HH_API_URL: string | undefined = process.env.HH_API_URL;
const HH_PROJECT: string = process.env.HH_PROJECT || "";

const tracer = await HoneyHiveTracer.init({
apiKey: HH_API_KEY,
project: HH_PROJECT,
sessionName: "my-session-name",
source: "my-source",
serverUrl: HH_API_URL,
});

Then you can use the traceFunction method to wrap any function you want to trace:

const myFunction = tracer.traceFunction()(
    function processParameters(param1: string, param2: number): string {
        // Your function code here
        return `Result with ${param1} and ${param2}`;
    }
);

// Call the traced function
myFunction("test", 42);

This will automatically create a span for the function, capturing its inputs and outputs.

Enriching Custom Spans

You can enrich your custom spans with additional properties to provide more context to your traces. There are 2 ways to enrich spans in HoneyHive:-

Enriching before execution: The decorator-based enrichment below shows how you can enrich static properties before the code executes.
Enriching during execution: You can enrich dynamic properties on a session/trace or event/span level using enrich_session and enrich_span functions. Please refer to the Enrich Traces documentation for more details on enrichment during execution.

Enriching Before Execution

You can provide additional configuration and metadata to your traced functions before execution:

interface Config {
important_setting: boolean;
}

interface Metadata {
version: string;
}

const config: Config = { important_setting: true };
const metadata: Metadata = { version: "1.0.0" };

const myConfiguredFunction = tracer.traceFunction({config: config, metadata: metadata})(
function processWithConfig(param1: string, param2: number): string {
    // Your function code here
    return `Result with ${param1} and ${param2}`;
}
);

The config and metadata parameters allow you to add custom attributes to the span, which can be useful for filtering and analyzing your traces later.

Enriching During Execution Using `enrich_span` or `enrich_session`

How it works

The traceFunction method performs the following actions:

Creates a new span with the function’s name (or ‘anonymous’ if the function is unnamed)
Captures all function inputs (parameters) as span attributes
Adds any provided config and metadata as span attributes
Executes the function
Captures the function’s return value as a span attribute
Ends the span

All captured data is automatically sent to your HoneyHive project, where you can view and analyze it alongside your other traces.

Handling asynchronous functions

The traceFunction method automatically handles both synchronous and asynchronous functions. When working with async functions, remember to:

Mark your function as async
Use await when calling the traced function

// Define an async traced function
const myAsyncFunction = tracer.traceFunction()(
    async function processAsync(param1: string): Promise<string> {
        // Async operations here
        const result = await someAsyncOperation(param1);
        return result;
    }
);

const result = await myAsyncFunction("test");

Event-Specific Tracing Functions

While traceFunction provides a general-purpose solution for tracing any function, HoneyHive also offers specialized tracing methods for different types of events: traceModel, traceTool, and traceChain. These methods are designed to trace specific operations in your application, such as LLM requests, deterministic functions, or pipelines, and automatically set the appropriate event type for the span.

traceModel

The traceModel method is used to trace operations involving LLM requests. It automatically sets the event type to model and allows you to capture inputs, outputs, and metadata specific to the model operation.

const modelFunction = tracer.traceModel(
    async function generateText(prompt: string): Promise<string> {
        const response = await LLMCompletion({ prompt });
        return response.text;
    },
    {
        metadata: { model: "gpt-4" },
        config: { temperature: 0.7 },
    }
);

In this example, the traceModel method captures the function’s inputs (e.g., the prompt), outputs (e.g., the generated text), and additional metadata (e.g., the model name and configuration).

traceTool

The traceTool method is used to trace deterministic operations, such as database queries, API requests, or regex parsing. It automatically sets the event type to tool and allows you to enrich the span with relevant metadata.

async function searchDatabase(query: string): Promise<any> {
    const result = await DatabaseSearch(query);
    return result;
}

// Trace the pre-defined function
const toolFunction = tracer.traceTool(searchDatabase, {
    metadata: { tool_type: "database_search" }
});

traceChain

The traceChain method is used to trace pipelines or workflows that group multiple model and tool events into a single composable unit. It automatically sets the event type to chain and allows you to capture the overall behavior of the pipeline.

const processingPipeline = async (input: string): Promise<string> => {
    const intermediateResult = await tracer.traceTool(FirstOperation, {
        metadata: { tool_type: "first_operation" },
    })(input);

    const finalResult = await tracer.traceTool(SecondOperation, {
        metadata: { tool_type: "second_operation" },
    })(intermediateResult);

    return finalResult;
};

const chainFunction = tracer.traceTool(processingPipeline, {
    metadata: { chain_name: "processing_pipeline" }
});

This method is particularly useful for monitoring and analyzing complex workflows, such as retrieval pipelines or post-processing pipelines.

Best practices

Use named functions when possible, as they will be used as span names in your traces
Be mindful of sensitive data when tracing functions - avoid capturing passwords or other secrets
Use the config parameter for settings that might affect the function’s behavior
Use the metadata parameter for additional context that could be useful for analysis

By strategically using the traceFunction, traceModel, traceTool, and traceChain methods on key functions in your application, you can gain valuable insights into your custom code’s performance and behavior, complementing HoneyHive’s automatic instrumentation of LLM components.

Visualizing Your Traces

Next Up: Enrich Your Traces

Here are some ways in which you can enrich your traces:

Enrichments Overview

An overview of the types of enrichments available for your traces and spans

Logging Client-Side Evaluations

Add client-side metrics, guardrail results, or evaluations to your traces

Logging User Feedback

Enrich traces with feedback or additional annotations

Logging Configuration Details

Include config details, model hyperparameters, or prompt templates

Logging User Properties

Add user properties or tracking IDs to your traces

Logging Custom Metadata

Enrich traces with arbitrary metadata or JSON

Introduction

Observability

Evaluation

Workspace Management

Learn More

Custom Spans

Prerequisites

Overview

Enriching Custom Spans

Enriching Before Execution Using Decorators

Enriching During Execution Using `enrich_span` or `enrich_session`

Async Function Tracing with @atrace

Best practices

Visualizing Your Traces

Next Up: Enrich Your Traces

Enrichments Overview

Logging Client-Side Evaluations

Logging User Feedback

Logging Configuration Details

Logging User Properties

Logging Custom Metadata

Using the `traceFunction` method

Enriching Custom Spans

Enriching Before Execution

Enriching During Execution Using `enrich_span` or `enrich_session`

How it works

Handling asynchronous functions

Event-Specific Tracing Functions

traceModel

traceTool

traceChain

Best practices

Visualizing Your Traces

Next Up: Enrich Your Traces

Enrichments Overview

Logging Client-Side Evaluations

Logging User Feedback

Logging Configuration Details

Logging User Properties

Logging Custom Metadata

Introduction

Observability

Evaluation

Workspace Management

Learn More

Documentation Index

​Prerequisites

​Overview

​Enriching Custom Spans

​Enriching Before Execution Using Decorators

​Enriching During Execution Using enrich_span or enrich_session

​Async Function Tracing with @atrace

​Best practices

​Visualizing Your Traces

​Next Up: Enrich Your Traces

Enrichments Overview

Logging Client-Side Evaluations

Logging User Feedback

Logging Configuration Details

Logging User Properties

Logging Custom Metadata

​Using the traceFunction method

​Enriching Custom Spans

​Enriching Before Execution

​Enriching During Execution Using enrich_span or enrich_session

​How it works

​Handling asynchronous functions

​Event-Specific Tracing Functions

​traceModel

​traceTool

​traceChain

​Best practices

​Visualizing Your Traces

​Next Up: Enrich Your Traces

Enrichments Overview

Logging Client-Side Evaluations

Logging User Feedback

Logging Configuration Details

Logging User Properties

Logging Custom Metadata

Prerequisites

Overview

Enriching Custom Spans

Enriching Before Execution Using Decorators

Enriching During Execution Using `enrich_span` or `enrich_session`

Async Function Tracing with @atrace

Best practices

Visualizing Your Traces

Next Up: Enrich Your Traces

Using the `traceFunction` method

Enriching Custom Spans

Enriching Before Execution

Enriching During Execution Using `enrich_span` or `enrich_session`

How it works

Handling asynchronous functions

Event-Specific Tracing Functions

traceModel

traceTool

traceChain

Best practices

Visualizing Your Traces

Next Up: Enrich Your Traces