This guide demonstrates how to build a sophisticated code generation system using LangGraph and HoneyHive tracing. The system combines Retrieval Augmented Generation (RAG) with self-correction capabilities to generate reliable code solutions. HoneyHive’s tracing capabilities provide comprehensive visibility into the entire process, making debugging and optimization easier than ever.

​
Overview

The system consists of several key components:

  • Documentation loading and processing (traced with HoneyHive)
  • LLM setup with structured output (monitored for performance)
  • A LangGraph workflow for code generation and validation (with detailed tracing)
  • HoneyHive tracing for monitoring and debugging (real-time insights)

​
Prerequisites

Before running this code, ensure you have the following:

  • Python 3.x
  • Required API keys:
    • HoneyHive API key (for comprehensive tracing)
    • OpenAI API key
    • Anthropic API key
  • Required packages: 
    pip install langchain langgraph beautifulsoup4 honeyhive langchain-openai langchain-anthropic

​
Code Implementation

​
Environment Setup and Imports

import getpass
import os
import sys
from bs4 import BeautifulSoup as Soup
from langchain_community.document_loaders.recursive_url_loader import RecursiveUrlLoader
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
from pydantic import BaseModel, Field
from typing import List
from typing_extensions import TypedDict, Annotated
from langgraph.graph import END, StateGraph, START
from honeyhive import HoneyHiveTracer, trace

# Set up environment variables
os.environ["HONEYHIVE_API_KEY"] = "your honeyhive api key"
os.environ["HONEYHIVE_PROJECT"] = "your honeyhive project"
os.environ["HONEYHIVE_SOURCE"] = "your honeyhive source"
os.environ["HONEYHIVE_SESSION_NAME"] = "your session name"
os.environ["OPENAI_API_KEY"] = "your openai api key"
os.environ["ANTHROPIC_API_KEY"] = "your anthropic api key"

​
HoneyHive Tracing Setup

HoneyHive’s tracing setup provides comprehensive monitoring capabilities:

# Initialize HoneyHive tracer with detailed configuration
HoneyHiveTracer.init(
    api_key=os.environ.get("HONEYHIVE_API_KEY", "your honeyhive api key"),
    project=os.environ.get("HONEYHIVE_PROJECT", "your honeyhive project"),
    source="development",
    session_name="LangGraph Code Generation"
)

With this setup, you get:

  • Real-time monitoring of all traced functions
  • Detailed performance metrics
  • Error tracking and debugging
  • Session-based analytics
  • Custom metadata support

​
Documentation Loading

The system uses RecursiveUrlLoader with HoneyHive tracing to monitor documentation loading:

@trace
def load_documentation(url):
    """Load documentation from a URL"""
    print("---LOADING DOCUMENTATION---")
    loader = RecursiveUrlLoader(
        url=url, max_depth=20, extractor=lambda x: Soup(x, "html.parser").text
    )
    docs = loader.load()

    # Sort and concatenate documentation
    d_sorted = sorted(docs, key=lambda x: x.metadata["source"])
    d_reversed = list(reversed(d_sorted))
    concatenated_content = "\n\n\n --- \n\n\n".join(
        [doc.page_content for doc in d_reversed]
    )
    print("---DOCUMENTATION LOADED---")
    return concatenated_content

HoneyHive tracing here provides:

  • Loading time metrics
  • Document count tracking
  • Error handling for failed loads
  • Memory usage monitoring
  • URL accessibility tracking

​
Data Model

The system uses Pydantic with HoneyHive tracing for structured output validation:

class code(BaseModel):
    """Schema for code solutions to questions about HoneyHive."""
    prefix: str = Field(description="Description of the problem and approach")
    imports: str = Field(description="Code block import statements")
    code: str = Field(description="Code block not including import statements")

HoneyHive helps track:

  • Model validation success rates
  • Field completion rates
  • Schema compliance
  • Data quality metrics

​
LLM Setup

The system uses Claude with HoneyHive tracing for comprehensive LLM monitoring:

code_gen_prompt_claude = ChatPromptTemplate.from_messages([
    (
        "system",
        """<instructions> You are a coding assistant with expertise in HoneyHive. \n 
        Here is the LCEL documentation:  \n ------- \n  {context} \n ------- \n Answer the user  question based on the \n 
        above provided documentation. Ensure any code you provide can be executed with all required imports and variables \n
        defined. Structure your answer: 1) a prefix describing the code solution, 2) the imports, 3) the functioning code block. \n
        Invoke the code tool to structure the output correctly. </instructions> \n Here is the user question:""",
    ),
    ("placeholder", "{messages}"),
])

@trace
def setup_llm():
    """Set up the LLM with structured output"""
    expt_llm_claude = "claude-3-7-sonnet-latest"
    llm_claude = ChatAnthropic(
        model=expt_llm_claude,
        default_headers={"anthropic-beta": "tools-2024-04-04"},
    )
    structured_llm_claude = llm_claude.with_structured_output(code, include_raw=True)
    return structured_llm_claude

HoneyHive provides:

  • LLM response time tracking
  • Token usage monitoring
  • Error rate tracking
  • Model performance analytics
  • Cost tracking per request

​
LangGraph Implementation

The system uses LangGraph with HoneyHive tracing for workflow monitoring:

  1. State Definition:
class GraphState(TypedDict):
    """Represents the state of our graph."""
    error: str
    messages: List
    generation: str
    iterations: int
  1. Graph Nodes with HoneyHive tracing:
  • generate: Monitors code generation performance
  • code_check: Tracks validation success rates
  • reflect: Measures improvement iterations
  • decide_to_finish: Tracks decision points
  1. Graph Construction with comprehensive tracing:
@trace
def build_graph():
    """Build the LangGraph for code generation"""
    graph_builder = StateGraph(GraphState)
    
    # Add nodes
    graph_builder.add_node("generate", generate)
    graph_builder.add_node("code_check", code_check)
    graph_builder.add_node("reflect", reflect)
    
    # Add edges
    graph_builder.add_edge(START, "generate")
    graph_builder.add_edge("generate", "code_check")
    
    # Add conditional edges
    graph_builder.add_conditional_edges(
        "code_check",
        decide_to_finish,
        {
            "generate": "generate",
            "reflect": "reflect",
            "finish": END,
        },
    )
    graph_builder.add_edge("reflect", "generate")
    
    return graph_builder.compile()

HoneyHive tracing provides:

  • Node execution time tracking
  • Edge traversal monitoring
  • State transition tracking
  • Error propagation analysis
  • Performance bottlenecks identification

​
Usage

To use the code generation system with HoneyHive monitoring:

if __name__ == "__main__":
    question = "How can I use HoneyHive tracing with LangGraph?"
    solution = solve_coding_question(question)
    
    print("\n=== FINAL SOLUTION ===")
    print(f"\n{solution.prefix}\n")
    print(f"IMPORTS:\n{solution.imports}\n")
    print(f"CODE:\n{solution.code}")

​
Key Features

  1. RAG Integration: HoneyHive traces document retrieval and processing
  2. Self-Correction: Monitors validation and improvement cycles
  3. Structured Output: Tracks schema compliance and data quality
  4. HoneyHive Tracing: Provides comprehensive monitoring and debugging
  5. Maximum Iterations: Tracks iteration counts and success rates

​
Best Practices

  1. Always set up proper environment variables before running
  2. Monitor the HoneyHive dashboard for:
    • Performance metrics
    • Error rates
    • Cost analysis
    • Usage patterns
  3. Adjust the max_depth parameter in RecursiveUrlLoader based on your needs
  4. Customize the reflection step based on your specific use case
  5. Implement proper error handling for production use

​
HoneyHive Dashboard Insights

The HoneyHive dashboard provides valuable insights:

  1. Performance Metrics:
    • Response times
    • Throughput
    • Resource usage
  2. Error Tracking:
    • Error rates
    • Stack traces
    • Error patterns
  3. Cost Analysis:
    • API usage costs
    • Resource consumption
    • Cost optimization opportunities
  4. Usage Patterns:
    • Peak usage times
    • Common operations
    • User behavior

​
Conclusion

This implementation provides a robust foundation for code generation with self-correction capabilities. The combination of LangGraph and HoneyHive tracing ensures reliable and monitored code generation processes. HoneyHive’s comprehensive tracing capabilities make it easier to:

  • Debug issues
  • Optimize performance
  • Track costs
  • Monitor quality
  • Scale the system

Was this page helpful?

Suggest editsRaise issue