Documentation Index
Fetch the complete documentation index at: https://docs.honeyhive.ai/llms.txt
Use this file to discover all available pages before exploring further.
Introduction
HoneyHive’s tracing functionality includes support for tracking configurations, prompt templates, and other LLM configs in your traces.
Prerequisites
You have already set tracing for your code as described in our quickstart guide.
Setting
You can set on both the trace level or the span level. If the applies to the entire trace, then set it on the trace level. If the applies to a specific span, then set it on the span level. For more details, refer to the enrich traces documentation.
HoneyHive automatically captures most model providers. Only use this function when you want to capture additional configs that are not automatically captured. You can find the full list of supported packages here. In Python, you can use the enrich_session function to set on the trace level.To pass to HoneyHive, pass it to the param in the enrich_session function. This function is used to enrich the session with additional information. Remember that enrich_session will update, not overwrite, the existing object on the trace.Read more about the enrich_session function in the Python SDK reference.Here’s an example of how to set on the trace level in Python:from honeyhive import HoneyHiveTracer, enrich_session
HoneyHiveTracer.init(
api_key="my-api-key",
project="my-project",
)
# ...
enrich_session(config={
"template": prompt_template["template"],
"prompt": prompt_template["prompt"],
"hyperparams": {
"temperature": 0.5,
"max_tokens": 100,
"top_p": 0.9,
"top_k": 50,
}
# optionally adding any arbitrary fields as you need
})
In Python, you can use the enrich_span function to set on the span level.To pass to HoneyHive, pass it to the param in the enrich_span function. This function is used to enrich the span with additional information. Remember that enrich_span will update, not overwrite, the existing object linked to the span.Read more about the enrich_span function in the Python SDK reference.Here’s an example of how to set on the span level in Python:from honeyhive import HoneyHiveTracer, trace, enrich_span
HoneyHiveTracer.init(
api_key="my-api-key",
project="my-project",
)
# ...
@trace
def my_function(input, prompt_template):
# ...
enrich_span(config={
"template": prompt_template["template"],
"prompt": prompt_template["prompt"],
"hyperparams": {
"temperature": 0.5,
"max_tokens": 100,
"top_p": 0.9,
"top_k": 50,
}
})
# ...
return response
# ...
In TypeScript, you can use the tracer.enrichSession function to set on the trace level.To pass to HoneyHive, pass it to the param in the tracer.enrichSession function. This function is used to enrich the session with additional information. Remember that tracer.enrichSession will update, not overwrite, the existing object linked to the trace.Read more about the tracer.enrichSession function in the TypeScript SDK reference.Here’s an example of how to set on the trace level in TypeScript:import { HoneyHiveTracer, enrichSession } from "honeyhive";
// Initialize tracer
// Ensure HH_API_KEY and HH_PROJECT are set in your environment
const tracer = await HoneyHiveTracer.init({
sessionName: "setting-config-session"
// apiKey and project will be picked from environment variables
});
// Define the configuration object
const prompt_template = {
template: "Translate the following text to French: {{text}}",
prompt: "Translate the following text to French: Hello world",
hyperparams: {
temperature: 0.5,
max_tokens: 100,
top_p: 0.9,
top_k: 50,
}
};
// Wrap the execution logic in tracer.trace()
await tracer.trace(async () => {
// Add configuration to the entire trace session
enrichSession({
config: prompt_template
});
// ... rest of your application logic ...
console.log("Trace session enriched with config.");
});
// await tracer.flush(); // If the script exits immediately
In TypeScript, you can use the tracer.enrichSpan function to set on the span level.To pass to HoneyHive, pass it to the param in the tracer.enrichSpan function. This function is used to enrich the span with additional information. Remember that tracer.enrichSpan will update, not overwrite, the existing object linked to the span.Read more about the tracer.enrichSpan function in the TypeScript SDK reference.Here’s an example of how to set on the span level in TypeScript:import { HoneyHiveTracer, traceTool, enrichSpan } from "honeyhive";
// Define types for the prompt template structure
interface PromptMessage {
role: string;
content: string;
}
interface PromptTemplate {
template: PromptMessage[];
prompt: PromptMessage[];
}
// Initialize tracer
// Ensure HH_API_KEY and HH_PROJECT are set in your environment
const tracer = await HoneyHiveTracer.init({
sessionName: "setting-config-session"
// apiKey and project will be picked from environment variables
});
// Define the function to be traced
const myTracedFunction = traceTool(
function my_function( // Function name is used as span name
input: string,
prompt_template: PromptTemplate
) {
// Add configuration specific to this span
enrichSpan({
config: {
template: prompt_template.template,
prompt: prompt_template.prompt,
hyperparams: {
temperature: 0.5,
max_tokens: 100,
top_p: 0.9,
top_k: 50,
}
}
});
// Your function code here
const response = `Processed input: ${input}`;
return response;
}
);
// --- Main Execution Logic ---
// Wrap the execution in tracer.trace() to establish context
await tracer.trace(async () => {
const prompt_template_data: PromptTemplate = {
template: [
{ role: "system", content: "You are a helpful AI assistant." },
{ role: "user", content: "Write a short poem about programming." }
],
prompt: [
{ role: "system", content: "You are a helpful AI assistant." },
{ role: "user", content: "Write a short poem about programming." }
]
};
// Execute the traced function within the trace context
myTracedFunction("Some input data", prompt_template_data);
});
// await tracer.flush(); // If the script exits immediately
Previously, tracing and enrichment involved calling methods directly on the tracer instance (e.g., tracer.traceFunction(), tracer.enrichSpan(), tracer.enrichSession()). While this pattern still works, it is now deprecated and will be removed in a future major version.Please update your code to use the imported functions (traceTool, enrichSpan, enrichSession) along with the tracer.trace() wrapper as shown in the examples above. This new approach simplifies usage within nested functions by not requiring the tracer instance to be passed around.Example of the deprecated pattern:// OLD (DEPRECATED) PATTERN:
// const tracer = await HoneyHiveTracer.init({...});
// tracer.enrichSession({ config: {...} });
// const myFunc = tracer.traceFunction()(function(...) { ... });
// tracer.enrichSpan({ config: {...} });
Concepts
Analyzing configurations in traces
By including configurations in your traces, you can:
- Track how different prompt structures affect your model’s output.
- Analyze the impact of specific placeholder values on performance.
- Compare prompts across different runs or sessions.
- Identify patterns in successful or unsuccessful prompts.
- Get more insights into how different models perform under the same conditions.
By incorporating configurations into your HoneyHive traces, you can gain deeper insights into how your prompts are constructed and how they perform, enabling more effective prompt engineering and optimization.
Learn More
SDK Reference
Read more about the enrich_span function in the Python SDK reference.
