Custom Spans

While the HoneyHive tracer automatically instruments all of the most common LLM application components, you might sometimes want to trace parts of your code beyond the automated coverage.

To do this, you can use HoneyHive’s manual tracing tools, which we’ll walk through in this guide.

​

Prerequisites

Please complete the Quickstart and make sure you have initialized your tracer using HoneyHiveTracer.init().

​

Overview

HoneyHive gives you full customizability over your traces.

You can trace any function in your code and see its inputs, outputs, errors, duration, etc. by decorating it with the @trace decorator.

from honeyhive import trace

@trace
def my_function(param1, param2):
    # Code here
    return result

​

Enriching Custom Spans

You can enrich your custom spans with additional properties to provide more context to your traces. There are 2 ways to enrich spans in HoneyHive:-

1. Enriching before execution: The decorator-based enrichment below shows how you can enrich static properties before the code executes.

2. Enriching during execution: You can enrich dynamic properties on a session/trace or event/span level using enrich_session and enrich_span functions. Please refer to the Enrich Traces documentation for more details on enrichment during execution.

​

Enriching Before Execution Using Decorators

Here is an example of how to enrich the Configuration, Metadata, and Event Type fields on the decorator:

from honeyhive import trace

@trace(
    config={
        "event_type": "my_event_type",
        "metadata": {"key": "value"},
    },
    metadata={
        "key": "value",
    },
    event_type="chain",
)
def my_function(param1, param2):
    # Code here
    return result

​

Enriching During Execution Using enrich_span or enrich_session

You can also enrich your traces and custom spans with various propertires during execution using the enrich_span or enrich_session functions.

Please refer to the Enrich Traces documentation for more details on enrichment during execution. You can find the complete documentation for this in the Python SDK reference.

​

Async Function Tracing with @atrace

For async functions in Python, use the @atrace decorator instead of @trace. The @atrace decorator is specifically designed to handle async functions properly:

from honeyhive import atrace

@atrace
async def my_async_function(param1: str) -> str:
    # Async operations here
    result = await some_async_operation(param1)
    return result

# The function can be called with await as normal
result = await my_async_function("test")

The @atrace decorator supports the same enrichment options as @trace:

@atrace(
    config={"event_type": "my_event_type"},
    metadata={"key": "value"},
    event_type="chain"
)
async def my_async_function(param1: str) -> str:
    result = await some_async_operation(param1)
    return result

​

Best practices

• Use descriptive function names, as they will be used as span names in your traces
• Be mindful of sensitive data when tracing functions - avoid capturing passwords or other secrets

By strategically adding the @trace decorator to key functions in your application, you can gain valuable insights into your custom code’s performance and behavior, complementing HoneyHive’s automatic instrumentation of LLM components.

​

Visualizing Your Traces

All traces can be visualized in the HoneyHive web app under Data Store in the session’s specified project. The session tree view shows you the execution flow of your trace, as well all the automatically captured and manually enriched attributes.

​

Next Up: Enrich Your Traces

Here are some ways in which you can enrich your traces:

While HoneyHive automatically instruments many common LLM application components, you may sometimes want to trace specific parts of your code beyond the automated coverage. HoneyHive provides flexible tracing methods including the general-purpose traceFunction as well as event-specific methods (traceModel, traceTool, traceChain) that allow you to easily add custom spans to your TypeScript application. These methods capture function inputs, outputs, and additional metadata, providing deeper insights into your application’s behavior.

​

Using the traceFunction method

To use the traceFunction method, you first need to initialize the HoneyHiveTracer:

import { HoneyHiveTracer } from "honeyhive";

const HH_API_KEY: string = process.env.HH_API_KEY || "";
const HH_API_URL: string | undefined = process.env.HH_API_URL;
const HH_PROJECT: string = process.env.HH_PROJECT || "";

const tracer = await HoneyHiveTracer.init({
apiKey: HH_API_KEY,
project: HH_PROJECT,
sessionName: "my-session-name",
source: "my-source",
serverUrl: HH_API_URL,
});

Then you can use the traceFunction method to wrap any function you want to trace:

const myFunction = tracer.traceFunction()(
    function processParameters(param1: string, param2: number): string {
        // Your function code here
        return `Result with ${param1} and ${param2}`;
    }
);

// Call the traced function
myFunction("test", 42);

This will automatically create a span for the function, capturing its inputs and outputs.

​

Enriching Custom Spans

You can enrich your custom spans with additional properties to provide more context to your traces. There are 2 ways to enrich spans in HoneyHive:-

1. Enriching before execution: The decorator-based enrichment below shows how you can enrich static properties before the code executes.

2. Enriching during execution: You can enrich dynamic properties on a session/trace or event/span level using enrich_session and enrich_span functions. Please refer to the Enrich Traces documentation for more details on enrichment during execution.

​

Enriching Before Execution

You can provide additional configuration and metadata to your traced functions before execution:

interface Config {
important_setting: boolean;
}

interface Metadata {
version: string;
}

const config: Config = { important_setting: true };
const metadata: Metadata = { version: "1.0.0" };

const myConfiguredFunction = tracer.traceFunction({config: config, metadata: metadata})(
function processWithConfig(param1: string, param2: number): string {
    // Your function code here
    return `Result with ${param1} and ${param2}`;
}
);

The config and metadata parameters allow you to add custom attributes to the span, which can be useful for filtering and analyzing your traces later.

​

Enriching During Execution Using enrich_span or enrich_session

You can also enrich your traces and custom spans with various propertires during execution using the enrich_span or enrich_session functions.

Please refer to the Enrich Traces documentation for more details on enrichment during execution. You can find the complete documentation for this in the TypeScript SDK reference.

​

How it works

The traceFunction method performs the following actions:

1. Creates a new span with the function’s name (or ‘anonymous’ if the function is unnamed)
2. Captures all function inputs (parameters) as span attributes
3. Adds any provided config and metadata as span attributes
4. Executes the function
5. Captures the function’s return value as a span attribute
6. Ends the span

All captured data is automatically sent to your HoneyHive project, where you can view and analyze it alongside your other traces.

​

Handling asynchronous functions

The traceFunction method automatically handles both synchronous and asynchronous functions. When working with async functions, remember to:

1. Mark your function as async
2. Use await when calling the traced function

// Define an async traced function
const myAsyncFunction = tracer.traceFunction()(
    async function processAsync(param1: string): Promise<string> {
        // Async operations here
        const result = await someAsyncOperation(param1);
        return result;
    }
);

const result = await myAsyncFunction("test");

​

Event-Specific Tracing Functions

While traceFunction provides a general-purpose solution for tracing any function, HoneyHive also offers specialized tracing methods for different types of events: traceModel, traceTool, and traceChain. These methods are designed to trace specific operations in your application, such as LLM requests, deterministic functions, or pipelines, and automatically set the appropriate event type for the span.

​

traceModel

The traceModel method is used to trace operations involving LLM requests. It automatically sets the event type to model and allows you to capture inputs, outputs, and metadata specific to the model operation.

const modelFunction = tracer.traceModel(
    async function generateText(prompt: string): Promise<string> {
        const response = await LLMCompletion({ prompt });
        return response.text;
    },
    {
        metadata: { model: "gpt-4" },
        config: { temperature: 0.7 },
    }
);

In this example, the traceModel method captures the function’s inputs (e.g., the prompt), outputs (e.g., the generated text), and additional metadata (e.g., the model name and configuration).

​

traceTool

The traceTool method is used to trace deterministic operations, such as database queries, API requests, or regex parsing. It automatically sets the event type to tool and allows you to enrich the span with relevant metadata.

async function searchDatabase(query: string): Promise<any> {
    const result = await DatabaseSearch(query);
    return result;
}

// Trace the pre-defined function
const toolFunction = tracer.traceTool(searchDatabase, {
    metadata: { tool_type: "database_search" }
});

​

traceChain

The traceChain method is used to trace pipelines or workflows that group multiple model and tool events into a single composable unit. It automatically sets the event type to chain and allows you to capture the overall behavior of the pipeline.

const processingPipeline = async (input: string): Promise<string> => {
    const intermediateResult = await tracer.traceTool(FirstOperation, {
        metadata: { tool_type: "first_operation" },
    })(input);

    const finalResult = await tracer.traceTool(SecondOperation, {
        metadata: { tool_type: "second_operation" },
    })(intermediateResult);

    return finalResult;
};

const chainFunction = tracer.traceTool(processingPipeline, {
    metadata: { chain_name: "processing_pipeline" }
});

This method is particularly useful for monitoring and analyzing complex workflows, such as retrieval pipelines or post-processing pipelines.

​

Best practices

• Use named functions when possible, as they will be used as span names in your traces
• Be mindful of sensitive data when tracing functions - avoid capturing passwords or other secrets
• Use the config parameter for settings that might affect the function’s behavior
• Use the metadata parameter for additional context that could be useful for analysis

By strategically using the traceFunction, traceModel, traceTool, and traceChain methods on key functions in your application, you can gain valuable insights into your custom code’s performance and behavior, complementing HoneyHive’s automatic instrumentation of LLM components.

​

Visualizing Your Traces

All traces can be visualized in the HoneyHive web app under Data Store in the session’s specified project. The session tree view shows you the execution flow of your trace, as well all the automatically captured and manually enriched attributes.

​

Next Up: Enrich Your Traces

Here are some ways in which you can enrich your traces: