Azure OpenAI HoneyHive Tracing Quickstart
This guide provides a comprehensive overview of tracing Azure OpenAI API calls using HoneyHive, with practical examples for different tracing scenarios.
Getting Started
Prerequisites
Before you begin, make sure you have:
- Python 3.8+
- An Azure OpenAI resource with API access
- A HoneyHive API key
Installation
Install the required packages:
pip install openai honeyhive pydantic
Environment Setup
Create a .env
file with your API credentials:
HONEYHIVE_API_KEY=your_honeyhive_api_key
AZURE_OPENAI_API_KEY=your_azure_openai_api_key
AZURE_OPENAI_ENDPOINT=https://your-endpoint.openai.azure.com
AZURE_OPENAI_API_VERSION=2023-07-01-preview
GPT4_DEPLOYMENT_NAME=your-gpt4-deployment-name
Basic Configuration
Here’s how to initialize HoneyHive tracing and the Azure OpenAI client:
import os
from openai import AzureOpenAI
from honeyhive import HoneyHiveTracer, trace
# Initialize HoneyHive tracer
HoneyHiveTracer.init(
api_key=os.getenv("HONEYHIVE_API_KEY"),
project="Azure-OpenAI-traces"
)
# Initialize Azure OpenAI client
client = AzureOpenAI(
api_version=os.getenv("AZURE_OPENAI_API_VERSION", "2023-07-01-preview"),
azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
)
Tracing Types
1. Basic Chat Completions
The simplest form of tracing captures basic chat completions with the Azure OpenAI API:
@trace
def basic_chat_completion():
"""Make a simple chat completion call to Azure OpenAI API."""
try:
# This call will be automatically traced by HoneyHive
response = client.chat.completions.create(
model="deployment-name", # Replace with your actual deployment name
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"}
],
temperature=0.7,
max_tokens=150
)
# Return the response content
return response.choices[0].message.content
except Exception as e:
# Errors will be captured in the trace
print(f"Error: {e}")
raise
2. Function Calling Traces
Trace function calling with tools and handling of tool responses:
@trace
def basic_function_calling():
"""
Demonstrate basic function calling with Azure OpenAI API.
The model will decide when to call the function based on the user query.
"""
# Define the tools (functions) the model can use
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather in a specified location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and country, e.g., 'San Francisco, CA' or 'Paris, France'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Default is celsius."
}
},
"required": ["location"]
}
}
}
]
# Make a request to the Azure OpenAI API
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What's the weather like in Paris today?"}
]
# This API call will be traced by HoneyHive
response = client.chat.completions.create(
model="deployment-name", # Replace with your actual deployment name
messages=messages,
tools=tools,
tool_choice="auto"
)
# Continue processing the response...
3. Structured Output Traces
Trace structured outputs using response formats:
@trace
def get_structured_json():
"""Get a structured JSON response using the response_format parameter."""
try:
response = client.chat.completions.create(
model="deployment-name", # Replace with your actual deployment name
messages=[
{"role": "system", "content": "You are a helpful assistant that provides weather information."},
{"role": "user", "content": "What's the weather like in New York today?"}
],
response_format={"type": "json_object"}
)
return response.choices[0].message.content
except Exception as e:
print(f"Error: {e}")
raise
You can also trace Pydantic model parsing:
@trace
def get_weather_structured_output(location: str):
"""Get structured weather information for a location using Pydantic."""
try:
completion = client.beta.chat.completions.parse(
model="deployment-name", # Replace with your actual deployment name
messages=[
{"role": "system", "content": "You are a helpful assistant that provides weather information."},
{"role": "user", "content": f"What's the weather like in {location} today?"}
],
response_format=WeatherInfo
)
# The parsed attribute contains the structured data
weather_info = completion.choices[0].message.parsed
return weather_info
except Exception as e:
print(f"Error: {e}")
raise
4. Multi-Turn Conversation Traces
Track conversations across multiple turns:
class Conversation:
"""
Class to manage a conversation with the Azure OpenAI API.
Each turn in the conversation is traced by HoneyHive.
"""
def __init__(self, system_message="You are a helpful assistant."):
self.messages = [{"role": "system", "content": system_message}]
self.turn_count = 0
@trace
def add_user_message(self, content):
"""Add a user message to the conversation and get the assistant's response."""
# Increment turn count
self.turn_count += 1
# Add user message to the conversation
self.messages.append({"role": "user", "content": content})
try:
# Get assistant response
response = client.chat.completions.create(
model="deployment-name", # Replace with your actual deployment name
messages=self.messages,
temperature=0.7,
max_tokens=150
)
# Process response...
Usage example:
@trace
def run_rich_conversation():
"""Run a multi-turn conversation with the assistant on various topics."""
# Initialize conversation with a broad system message
conversation = Conversation(
system_message="You are a knowledgeable assistant able to discuss a wide range of topics."
)
# First turn
turn1 = conversation.add_user_message("Can you tell me about the Apollo 11 mission?")
# Second turn
turn2 = conversation.add_user_message("What were the names of the astronauts on that mission?")
# Third turn
turn3 = conversation.add_user_message("Let's switch topics. Can you explain how photosynthesis works?")
# And so on...
5. Reasoning Model Traces
Trace model behavior for complex reasoning tasks with temperature control:
@trace
def call_reasoning_model_math():
"""
Demonstrate calling a reasoning-capable model for math problems and trace the request/response.
Note: Use your Azure OpenAI deployed model that supports advanced reasoning.
"""
try:
# Complex math problem that benefits from reasoning capability
response = client.chat.completions.create(
model="gpt-4-deployment", # Replace with your actual GPT-4 deployment name
messages=[
{"role": "system", "content": "You are a helpful math assistant."},
{"role": "user", "content": "Solve this step by step: Integrate x^3 * ln(x) with respect to x."}
],
temperature=0.1 # Lower temperature for more precise reasoning
)
# Extract the response and the usage information
content = response.choices[0].message.content
return {
"content": content,
"usage": {
"prompt_tokens": response.usage.prompt_tokens,
"completion_tokens": response.usage.completion_tokens,
"total_tokens": response.usage.total_tokens
}
}
except Exception as e:
print(f"Error: {e}")
raise
Conclusion
HoneyHive provides comprehensive observability for your Azure OpenAI applications, allowing you to monitor API usage, performance, and quality. By integrating HoneyHive tracing into your Azure OpenAI applications, you can:
- Debug issues more effectively
- Optimize token usage
- Improve response quality
- Monitor application performance
- Track user interactions
For more information, refer to the HoneyHive Documentation and Azure OpenAI Documentation. Happy tracing!