HoneyHive Docs

Introduction

HoneyHive’s tracing functionality includes support for tracking configurations, prompt templates, and other LLM configs in your traces.

Prerequisites

You have already set tracing for your code as described in our quickstart guide.

Setting

You can set on both the trace level or the span level. If the applies to the entire trace, then set it on the trace level. If the applies to a specific span, then set it on the span level. For more details, refer to the enrich traces documentation.

HoneyHive automatically captures most model providers. Only use this function when you want to capture additional configs that are not automatically captured. You can find the full list of supported packages here.

In Python, you can use the enrich_session function to set on the trace level.

To pass to HoneyHive, pass it to the param in the enrich_session function. This function is used to enrich the session with additional information. Remember that enrich_session will update, not overwrite, the existing object on the trace.

Read more about the enrich_session function in the Python SDK reference.

Here’s an example of how to set on the trace level in Python:

Python
from honeyhive import HoneyHiveTracer, enrich_session

HoneyHiveTracer.init(
  api_key="my-api-key",
  project="my-project",
)

# ...

enrich_session(config={
  "template": prompt_template["template"],
  "prompt": prompt_template["prompt"],
  "hyperparams": {
      "temperature": 0.5,
      "max_tokens": 100,
      "top_p": 0.9,
      "top_k": 50,
  }
  # optionally adding any arbitrary fields as you need
})

In Python, you can use the enrich_session function to set on the trace level.

Read more about the enrich_session function in the Python SDK reference.

Here’s an example of how to set on the trace level in Python:

Python
from honeyhive import HoneyHiveTracer, enrich_session

HoneyHiveTracer.init(
  api_key="my-api-key",
  project="my-project",
)

# ...

enrich_session(config={
  "template": prompt_template["template"],
  "prompt": prompt_template["prompt"],
  "hyperparams": {
      "temperature": 0.5,
      "max_tokens": 100,
      "top_p": 0.9,
      "top_k": 50,
  }
  # optionally adding any arbitrary fields as you need
})

In Python, you can use the enrich_span function to set on the span level.

To pass to HoneyHive, pass it to the param in the enrich_span function. This function is used to enrich the span with additional information. Remember that enrich_span will update, not overwrite, the existing object linked to the span.

Read more about the enrich_span function in the Python SDK reference.

Here’s an example of how to set on the span level in Python:

Python
from honeyhive import HoneyHiveTracer, trace, enrich_span

HoneyHiveTracer.init(
  api_key="my-api-key",
  project="my-project",
)

# ...

@trace
def my_function(input, prompt_template):
    # ...

    enrich_span(config={
        "template": prompt_template["template"],
        "prompt": prompt_template["prompt"],
        "hyperparams": {
            "temperature": 0.5,
            "max_tokens": 100,
            "top_p": 0.9,
            "top_k": 50,
        }
    })

    # ...

    return response

# ...

In Python, you can use the enrich_session function to set on the trace level.

Read more about the enrich_session function in the Python SDK reference.

Here’s an example of how to set on the trace level in Python:

Python
from honeyhive import HoneyHiveTracer, enrich_session

HoneyHiveTracer.init(
  api_key="my-api-key",
  project="my-project",
)

# ...

enrich_session(config={
  "template": prompt_template["template"],
  "prompt": prompt_template["prompt"],
  "hyperparams": {
      "temperature": 0.5,
      "max_tokens": 100,
      "top_p": 0.9,
      "top_k": 50,
  }
  # optionally adding any arbitrary fields as you need
})

In Python, you can use the enrich_session function to set on the trace level.

Read more about the enrich_session function in the Python SDK reference.

Here’s an example of how to set on the trace level in Python:

Python
from honeyhive import HoneyHiveTracer, enrich_session

HoneyHiveTracer.init(
  api_key="my-api-key",
  project="my-project",
)

# ...

enrich_session(config={
  "template": prompt_template["template"],
  "prompt": prompt_template["prompt"],
  "hyperparams": {
      "temperature": 0.5,
      "max_tokens": 100,
      "top_p": 0.9,
      "top_k": 50,
  }
  # optionally adding any arbitrary fields as you need
})

In Python, you can use the enrich_span function to set on the span level.

Read more about the enrich_span function in the Python SDK reference.

Here’s an example of how to set on the span level in Python:

Python
from honeyhive import HoneyHiveTracer, trace, enrich_span

HoneyHiveTracer.init(
  api_key="my-api-key",
  project="my-project",
)

# ...

@trace
def my_function(input, prompt_template):
    # ...

    enrich_span(config={
        "template": prompt_template["template"],
        "prompt": prompt_template["prompt"],
        "hyperparams": {
            "temperature": 0.5,
            "max_tokens": 100,
            "top_p": 0.9,
            "top_k": 50,
        }
    })

    # ...

    return response

# ...

In TypeScript, you can use the tracer.enrichSession function to set on the trace level.

To pass to HoneyHive, pass it to the param in the tracer.enrichSession function. This function is used to enrich the session with additional information. Remember that tracer.enrichSession will update, not overwrite, the existing object linked to the trace.

Read more about the tracer.enrichSession function in the TypeScript SDK reference.

Here’s an example of how to set on the trace level in TypeScript:

TypeScript
import { HoneyHiveTracer, enrichSession } from "honeyhive";

// Initialize tracer 
// Ensure HH_API_KEY and HH_PROJECT are set in your environment
const tracer = await HoneyHiveTracer.init({
    sessionName: "setting-config-session"
    // apiKey and project will be picked from environment variables
});

// Define the configuration object
const prompt_template = {
    template: "Translate the following text to French: {{text}}",
    prompt: "Translate the following text to French: Hello world",
    hyperparams: {
        temperature: 0.5,
        max_tokens: 100,
        top_p: 0.9,
        top_k: 50,
    }
};

// Wrap the execution logic in tracer.trace()
await tracer.trace(async () => {
    // Add configuration to the entire trace session
    enrichSession({
      config: prompt_template
    });

    // ... rest of your application logic ...
    console.log("Trace session enriched with config.");
});

// await tracer.flush(); // If the script exits immediately

In TypeScript, you can use the tracer.enrichSession function to set on the trace level.

Read more about the tracer.enrichSession function in the TypeScript SDK reference.

Here’s an example of how to set on the trace level in TypeScript:

TypeScript
import { HoneyHiveTracer, enrichSession } from "honeyhive";

// Initialize tracer 
// Ensure HH_API_KEY and HH_PROJECT are set in your environment
const tracer = await HoneyHiveTracer.init({
    sessionName: "setting-config-session"
    // apiKey and project will be picked from environment variables
});

// Define the configuration object
const prompt_template = {
    template: "Translate the following text to French: {{text}}",
    prompt: "Translate the following text to French: Hello world",
    hyperparams: {
        temperature: 0.5,
        max_tokens: 100,
        top_p: 0.9,
        top_k: 50,
    }
};

// Wrap the execution logic in tracer.trace()
await tracer.trace(async () => {
    // Add configuration to the entire trace session
    enrichSession({
      config: prompt_template
    });

    // ... rest of your application logic ...
    console.log("Trace session enriched with config.");
});

// await tracer.flush(); // If the script exits immediately

In TypeScript, you can use the tracer.enrichSpan function to set on the span level.

To pass to HoneyHive, pass it to the param in the tracer.enrichSpan function. This function is used to enrich the span with additional information. Remember that tracer.enrichSpan will update, not overwrite, the existing object linked to the span.

Read more about the tracer.enrichSpan function in the TypeScript SDK reference.

Here’s an example of how to set on the span level in TypeScript:

TypeScript
import { HoneyHiveTracer, traceTool, enrichSpan } from "honeyhive";

// Define types for the prompt template structure
interface PromptMessage {
    role: string;
    content: string;
}

interface PromptTemplate {
    template: PromptMessage[];
    prompt: PromptMessage[];
}

// Initialize tracer 
// Ensure HH_API_KEY and HH_PROJECT are set in your environment
const tracer = await HoneyHiveTracer.init({
    sessionName: "setting-config-session"
    // apiKey and project will be picked from environment variables
});

// Define the function to be traced
const myTracedFunction = traceTool(
    function my_function( // Function name is used as span name
        input: string, 
        prompt_template: PromptTemplate
    ) {
        // Add configuration specific to this span
        enrichSpan({
          config: {
            template: prompt_template.template,
            prompt: prompt_template.prompt,
            hyperparams: {
                temperature: 0.5,
                max_tokens: 100,
                top_p: 0.9,
                top_k: 50,
            }
          }
        });

        // Your function code here
        const response = `Processed input: ${input}`;
        return response;
    }
);

// --- Main Execution Logic ---
// Wrap the execution in tracer.trace() to establish context
await tracer.trace(async () => {
    const prompt_template_data: PromptTemplate = {
        template: [
            { role: "system", content: "You are a helpful AI assistant." },
            { role: "user", content: "Write a short poem about programming." }
        ],
        prompt: [
            { role: "system", content: "You are a helpful AI assistant." },
            { role: "user", content: "Write a short poem about programming." }
        ]
    };

    // Execute the traced function within the trace context
    myTracedFunction("Some input data", prompt_template_data);
});

// await tracer.flush(); // If the script exits immediately

Previously, tracing and enrichment involved calling methods directly on the tracer instance (e.g., tracer.traceFunction(), tracer.enrichSpan(), tracer.enrichSession()). While this pattern still works, it is now deprecated and will be removed in a future major version.

Please update your code to use the imported functions (traceTool, enrichSpan, enrichSession) along with the tracer.trace() wrapper as shown in the examples above. This new approach simplifies usage within nested functions by not requiring the tracer instance to be passed around.

Example of the deprecated pattern:

// OLD (DEPRECATED) PATTERN:
// const tracer = await HoneyHiveTracer.init({...});
// tracer.enrichSession({ config: {...} }); 
// const myFunc = tracer.traceFunction()(function(...) { ... });
// tracer.enrichSpan({ config: {...} }); 

Concepts

Analyzing configurations in traces

By including configurations in your traces, you can:

Track how different prompt structures affect your model’s output.
Analyze the impact of specific placeholder values on performance.
Compare prompts across different runs or sessions.
Identify patterns in successful or unsuccessful prompts.
Get more insights into how different models perform under the same conditions.

By incorporating configurations into your HoneyHive traces, you can gain deeper insights into how your prompts are constructed and how they perform, enabling more effective prompt engineering and optimization.

Learn More

SDK Reference

Read more about the enrich_span function in the Python SDK reference.

Introduction

Guides

Tutorials

Learn more

Configurations

Introduction

Prerequisites

Setting

Concepts

Analyzing configurations in traces

Learn More

SDK Reference

Introduction

Guides

Tutorials

Learn more

​Introduction

​Prerequisites

​Setting

​Concepts

​Analyzing configurations in traces

​Learn More

​SDK Reference

Introduction

Prerequisites

Setting

Concepts

Analyzing configurations in traces

Learn More

SDK Reference