Product

​

Python SDK

​

v1.0.0rc19

• Export operations (export(), export_async(), get_by_session_id()) no longer time out on large result sets. The default read timeout is now 5 minutes instead of 5 seconds.
• New HH_EXPORT_TIMEOUT_SECONDS environment variable to override the default export timeout for extremely large exports or constrained network environments

​

Python SDK

​

v1.0.0rc18

• Large query batching: list operations like datapoints.list() and experiments.list_runs() now automatically batch when exceeding 100 items, merging results transparently. No API changes needed, existing code works as before without silent truncation or HTTP errors on large lists.
• SDK identification headers (hh-sdk-version, hh-sdk-language, hh-sdk-package) sent on all HTTP requests for better debugging and support diagnostics

​

Core Platform

​

Introducing Workspaces

​

Workspace-Scoped AI Provider Keys

• Smaller blast radius - A compromised key only affects one workspace, not your entire organization. Workspace Admins manage their own keys without needing org-level access.
• Cost attribution by team - Each workspace uses its own provider keys, so API spend maps directly to the business unit, department, or team that owns the workspace. No more splitting a shared org bill across teams.
• Right provider for the right team - One team can use Azure OpenAI to meet compliance requirements while another runs on AWS Bedrock. Each workspace’s provider configuration is fully independent.

​

Enterprise-Grade Role-Based Access Control

​

Introducing Organization Templates

​

LLM Evaluators with Workspace Provider Defaults

​

Other Improvements

• Visibility toggle on input fields and improved dialog interactions
• ClickHouse query performance improvements
• Log Store UI fixes
• Security-related fixes
• Form validation improvements

​

Python SDK

​

v1.0.0rc17

• Git context automatically stamped on experiment runs (commit hash, branch, author, remote URL, dirty status)
• Custom run_id support in evaluate() for using your own identifiers
• Auto-infer is_evaluation=True when dataset_id, datapoint_id, and run_id are all provided, preventing silent loss of evaluation context
• export_async() now retries on transient HTTP errors (502, 503, 504), matching export() behavior
• Debug logging for get_by_session_id and export flows (entry/exit, HTTP metadata, empty result diagnostics)
• Removed non-functional include_datapoints query param
• Synced OpenAPI spec and removed defunct Tools API client
• Fixed single-item list query param serialization causing 400 errors

​

v1.0.0rc16

• Fixed import errors and AttributeError in events and context modules
• Events API fixes: ordering, project deprecation handling, enrich_span event ID resolution
• Corrected evaluate() docstrings to match actual function signature

​

v1.0.0rc15

• New flush() method on HoneyHiveTracer for explicitly flushing pending spans before application shutdown
• New get_by_session_id() method on Events API for retrieving all events for a given session using the Data Plane export endpoint
• enrich_span() now supports update_event_id parameter for overriding a span’s event_id attribute without making API calls
• Events returned in chronological order by default; project parameter deprecated in favor of project-scoped API keys
• Pydantic models now preserve extra fields from API responses (extra="allow"), preventing data loss when backend returns additional fields

​

v1.0.0rc9

• Auto-generated type-safe v1 API client from OpenAPI spec with full Pydantic models for all requests/responses, sync and async methods, and new endpoint support (batch events, experiment results/comparison, project CRUD)
• Backwards-compatible API aliases: all API classes support both new (list(), create(), get()) and legacy (list_datasets(), create_dataset()) method names
• OTLP HTTP/JSON export is now the default format (changed from http/protobuf), configurable via HH_OTLP_PROTOCOL or OTEL_EXPORTER_OTLP_PROTOCOL environment variables
• evaluate() now accepts an instrumentors parameter to auto-instrument third-party libraries (OpenAI, Anthropic, Google ADK, LangChain, etc.) per datapoint
• evaluate() now accepts async functions, automatically detected and executed with asyncio.run() inside worker threads
• Typed Pydantic models for experiment results (MetricDetail, DatapointResult, DatapointMetric)
• Fixed metrics table printing empty values after evaluate() by aligning with backend’s details array format
• Fixed enrich_session() silently failing when called without explicit parameters

​

Python SDK

​

v1.0.0rc5

• Metric schema updated for backend parity: new type enums (PYTHON/LLM/HUMAN/COMPOSITE), categorical return type, and new fields (sampling_percentage, scale, categories, filters)
• Enhanced datapoints filtering with dataset_id and dataset_name parameters; legacy dataset parameter auto-detects ID vs. name
• EventsAPI.list_events() now accepts a single EventFilter or a list, converting automatically
• Simplified distributed tracing setup with with_distributed_trace_context() context manager (reduces server-side boilerplate from ~65 lines to 1 line)
• Fixed @trace decorator overwriting distributed trace baggage (session_id, project, source) instead of preserving it
• Configurable OpenTelemetry span limits via TracerConfig or environment variables (HH_MAX_ATTRIBUTES, HH_MAX_EVENTS, HH_MAX_LINKS)
• Automatic preservation of critical HoneyHive attributes (session_id, event_type, event_name, source) when spans approach the attribute limit
• Fixed session ID initialization when explicitly providing a session_id to HoneyHiveTracer.init()

​

Core Platform

​

Experiments Dashboard

​

Annotation Queues

​

Improved Evaluators UX

​

New Evaluator Templates

​

Core Platform

​

Improved Review Mode

​

Categorical Evaluators

​

Core Platform

​

Thread View

​

Improved Graph View

​

Introducing Alerts

1. Comprehensive Monitoring: Track performance metrics (latency, error rate), quality scores from evaluators, cost and usage patterns, plus any custom fields from your events or sessions. Get visibility into what matters most for your AI applications.
2. Smart Alert Types: Aggregate Alerts trigger when metrics cross absolute thresholds, while Drift Alerts detect when current performance deviates from previous periods by a configurable percentage. Choose the right detection method for your use case.
3. Flexible Scheduling: Configure alerts to run hourly, daily, weekly, or monthly based on your monitoring needs. Set custom evaluation windows to balance responsiveness with noise reduction.
4. Streamlined Workflow: Real-time preview charts show exactly what your alert will monitor, guided configuration in the right panel walks you through setup, and a recent activity feed tracks alert history. Manage alert states (Active, Triggered, Resolved, Paused, Muted) directly from each alert’s detail page.

​

Evaluator Templates Gallery

​

Core Platform

​

New Trace Visualization Modes

1. Session Summaries and New Tree View: Unified view of metrics, evaluations, and feedback across all spans in an agent session. Get a comprehensive overview without jumping between individual spans to understand overall session performance.
   
2. Timeline View: Flamegraph visualization that identifies latency bottlenecks and shows the relationship between sequential and parallel operations in your agent workflows. Perfect for performance optimization and understanding execution flow.
   
3. Graph View: Visual representation of complex execution paths and decision points through multi-agent workflows. Quickly understand how your agents interact and make decisions at a glance.
   

​

Improved Log Store Analytics

​

Core Platform

​

Role-Based Access Control (RBAC)

1. Two-Tier Permission Structure: Granular permission management with organization and project-level controls. Organization Admins have full control across the entire organization, while Project Admins maintain complete control within specific projects. This creates clear boundaries between teams and prevents data leakage between business units.
2. Enhanced API Key Security: Project-specific API key scoping ensures that teams can only access data within their designated projects. This provides better security isolation and compliance with industry regulations, especially critical for organizations in financial services, healthcare, and insurance.
3. Flexible Team Management: Easy onboarding and role transitions with transparent permission hierarchy. Delegate administrative responsibilities without compromising security, and manage team member access as organizations evolve.
4. Seamless Migration Process: Existing customers can migrate to RBAC with minimal disruption. All current users are automatically assigned Organization Admin roles, and project-specific API keys are available in Settings. Legacy API keys will remain functional until August 31st, 2025.

​

Core Platform

• Added list of allowed characters for project names

​

Python SDK (Logger)

​

HoneyHive Logger (honeyhive-logger) released

• The logger sdk has
  1. No external dependencies
  2. A fully stateless design
• Optimized for
  • Serverless environments
  • Highly regulated environments with strict security requirements

​

TypeScript SDK (Logger)

​

HoneyHive Logger (@honeyhive/logger) released

• The logger sdk has
  1. No external dependencies
  2. A fully stateless design
• Optimized for
  • Serverless environments
  • Highly regulated environments with strict security requirements

​

Python SDK - Version [v0.2.49]

• Added type annotation to decorators and the evaluation harness

​

Documentation

• Added documentation for Python/Typescript Loggers
• Updated gemini integration documentation to use latest sdk (Python and TypeScript)

​

Core Platform

​

Support for External Datasets in Experiments

{ 
"id": "<CUSTOM_DATASET_ID>",  // Optional 
"name": "<DATASET_NAME>",     // Optional
"data": [ 
    { 
        "id": "<CUSTOM_DATAPOINT_ID_1>",  // Optional 
        "inputs": { ... }, 
        "ground_truths": { ... }, 
    }
    // Additional datapoints...
] 
}

• Bug fixes and improvements across various areas to enhance performance and stability.
• Bug fixes for playground & evaluator version controls.

​

Documentation

• Standardizes parameter names and clarified evaluation order in Experiments Quickstart and Python/TS SDK docs.
• Adds cookbook: Inspirational Quotes Recommender with Qdrant and OpenAI
• Adds an “Evaluating External Logs” tutorial.
• Updates Python and TypeScript SDK’s references and overall documentation to align with recent improvements and best practices.
• Adds Datasets Introduction Guide.
• Adds Server-side Evaluator Templates List documentation.
• Adds LangGraph integration documentation.

​

Core Platform

​

Wide Mode

• Expanded workspace area for a more immersive viewing experience
• Distraction-free environment when focusing on complex tasks
• Better content visibility on smaller screens and split-window setups
• Toggle controls accessible via the header menu for easy switching

​

Improved Experiments Layout

• Structured input visualization with collapsible sections
• Clear side-by-side metrics display for easier model comparison
• Improved performance statistics with visual rating indicators

​

Introducing Review Mode

• Tag traces with annotations from your Human Evaluators definitions
• Apply your custom criteria right in the UI
• Add comments when something interesting pops up

• Enhanced filter functionality: Added the ability to edit filters and improved schema discovery within filters.
• Fixed pagination issue for events table.
• Bug fixes and stability improvements for filtering functionality.
• Added support for exists and not exists operators in filters.
• Frontend styling improvements to enhance the user interface.
• Bug fixes and stability enhancements for a smoother user experience.

​

Python SDK - Version [v0.2.44]

• Improved error tracking for the tracer: Enhanced the capture of error messages for custom-decorated functions.
• Git context enrichment: Added support for capturing Git branch status in traces and experiments.
• Introduced the disable_http_tracing parameter during tracer initialization to disable HTTP event tracing.
• Fixed the traceloop version to 0.30.0 to resolve protobuf dependency conflicts.

​

Python SDK - Version [v0.2.36]

• Reduced package size for AWS lambda usage
• Removed Langchain dependency. For using Langchain callbacks, install Langchain separately
• Add lambda, core, and eval poetry installation groups

​

TypeScript SDK - Version [v1.0.33]

• Improved error tracking for the tracer: Enhanced the capture of error messages for traced functions.
• Git context enrichment: Added support for capturing Git branch status in traces and experiments.
• Introduced the disableHttpTracing parameter during tracer initialization to disable HTTP event tracing.

​

TypeScript SDK - Version [v1.0.23]

• Reduced package size for AWS lambda usage
• Disabled CommonJS autotracing 3rd party packages: Anthropic, Bedrock, Pinecone, ChromaDB, Cohere, Langchain, LlamaIndex, OpenAI. Please use custom tracing for instrumenting Typescript.
• Refactor custom tracer for better initialization syntax and using typescript

​

Documentation

• Improved documentation for async function handling.
• Added integration documentation for model providers:
  • Openai.
  • Azure OpenAI.
  • AWS Bedrock.
• Added a tutorial for running experiments with multi-step LLM applications with MongoDB and OpenAI.
• Adds Streamlit Cookbook for tracing model calls with collected user feedback on AI response.
• Standardized all JavaScript/TypeScript code examples to TypeScript across the documentation.
• Added troubleshooting guidance for SSL validation failures.
• Documented the disable_http_tracing/disableHttpTracing parameter in the SDK Reference.
• Removed references to init_from_session_id in favor of using init with the session_id parameter.
• Updated the observability tutorial/cookbook to use enrichSession instead of setFeedback/setMetadata
• Integrations - added CrewAI Integration documentation.
• Added schema documentation (now part of Enrichment Schema) to describe our schemas in detail including a list of reserved properties.
• Added Client-side Evaluators documentation to describe the use of client-side evaluators for both tracing and experiments
• Updated Custom Spans documentation to add reference to tracing methods traceModel/traceTool/traceChain (TypeScript)
• Integrations - added LanceDB Integration documentation
• Integrations - added Zilliz Integration documentation