Product Updates

​

Core Platform

​

Experiments Dashboard

​

Annotation Queues

​

Core Platform

​

Improved Evaluators UX

​

Core Platform

​

New Evaluator Templates

​

Core Platform

​

Improved Review Mode

​

Core Platform

​

Categorical Evaluators

​

Core Platform

​

Thread View

​

Core Platform

​

Improved Graph View

​

Core Platform

​

Introducing Alerts

1. Comprehensive Monitoring: Track performance metrics (latency, error rate), quality scores from evaluators, cost and usage patterns, plus any custom fields from your events or sessions. Get visibility into what matters most for your AI applications.
2. Smart Alert Types: Aggregate Alerts trigger when metrics cross absolute thresholds, while Drift Alerts detect when current performance deviates from previous periods by a configurable percentage. Choose the right detection method for your use case.
3. Flexible Scheduling: Configure alerts to run hourly, daily, weekly, or monthly based on your monitoring needs. Set custom evaluation windows to balance responsiveness with noise reduction.
4. Streamlined Workflow: Real-time preview charts show exactly what your alert will monitor, guided configuration in the right panel walks you through setup, and a recent activity feed tracks alert history. Manage alert states (Active, Triggered, Resolved, Paused, Muted) directly from each alert’s detail page.

​

Evaluator Templates Gallery

​

Core Platform

​

New Trace Visualization Modes

1. Session Summaries and New Tree View: Unified view of metrics, evaluations, and feedback across all spans in an agent session. Get a comprehensive overview without jumping between individual spans to understand overall session performance.
   
2. Timeline View: Flamegraph visualization that identifies latency bottlenecks and shows the relationship between sequential and parallel operations in your agent workflows. Perfect for performance optimization and understanding execution flow.
   
3. Graph View: Visual representation of complex execution paths and decision points through multi-agent workflows. Quickly understand how your agents interact and make decisions at a glance.
   

​

Improved Log Store Analytics

​

Core Platform

​

Role-Based Access Control (RBAC)

1. Two-Tier Permission Structure: Granular permission management with organization and project-level controls. Organization Admins have full control across the entire organization, while Project Admins maintain complete control within specific projects. This creates clear boundaries between teams and prevents data leakage between business units.
2. Enhanced API Key Security: Project-specific API key scoping ensures that teams can only access data within their designated projects. This provides better security isolation and compliance with industry regulations, especially critical for organizations in financial services, healthcare, and insurance.
3. Flexible Team Management: Easy onboarding and role transitions with transparent permission hierarchy. Delegate administrative responsibilities without compromising security, and manage team member access as organizations evolve.
4. Seamless Migration Process: Existing customers can migrate to RBAC with minimal disruption. All current users are automatically assigned Organization Admin roles, and project-specific API keys are available in Settings. Legacy API keys will remain functional until August 31st, 2025.

​

Core Platform

• Added list of allowed characters for project names

​

Python SDK (Logger)

​

HoneyHive Logger (honeyhive-logger) released

• The logger sdk has
  1. No external dependencies
  2. A fully stateless design
• Optimized for
  • Serverless environments
  • Highly regulated environments with strict security requirements

​

TypeScript SDK (Logger)

​

HoneyHive Logger (@honeyhive/logger) released

• The logger sdk has
  1. No external dependencies
  2. A fully stateless design
• Optimized for
  • Serverless environments
  • Highly regulated environments with strict security requirements

​

Python SDK - Version [v0.2.49]

• Added type annotation to decorators and the evaluation harness

​

Documentation

• Added documentation for Python/Typescript Loggers
• Updated gemini integration documentation to use latest sdk (Python and TypeScript)

​

Core Platform

​

Support for External Datasets in Experiments

{ 
"id": "<CUSTOM_DATASET_ID>",  // Optional 
"name": "<DATASET_NAME>",     // Optional
"data": [ 
    { 
        "id": "<CUSTOM_DATAPOINT_ID_1>",  // Optional 
        "inputs": { ... }, 
        "ground_truths": { ... }, 
    }
    // Additional datapoints...
] 
}

​

Documentation

• Standardizes parameter names and clarified evaluation order in Experiments Quickstart and Python/TS SDK docs.
• Adds cookbook: Inspirational Quotes Recommender with Qdrant and OpenAI

​

Core Platform

• Bug fixes and improvements across various areas to enhance performance and stability.

​

Documentation

• Adds Evaluating External Logs tutorial.
• Updates Python and TypeScript SDK’s references and overall documentation to align with recent improvements and best practices.

​

Core Platform

• Bug fixes for playground & evaluator version controls.

​

Documentation

• Adds Datasets Introduction Guide.
• Adds Server-side Evaluator Templates List documentation.
• Adds LangGraph Integration documentation.

​

Core Platform

​

Wide Mode

• Expanded workspace area for a more immersive viewing experience
• Distraction-free environment when focusing on complex tasks
• Better content visibility on smaller screens and split-window setups
• Toggle controls accessible via the header menu for easy switching

​

Improved Experiments Layout

• Structured input visualization with collapsible sections
• Clear side-by-side metrics display for easier model comparison
• Improved performance statistics with visual rating indicators

• Bug fixes and stability improvements for filtering functionality.
• Added support for exists and not exists operators in filters.
• Frontend styling improvements to enhance the user interface.
• Bug fixes and stability enhancements for a smoother user experience.

​

Documentation

• Improved documentation for async function handling.
• Added integration documentation for model providers:
  • Openai..
  • Azure OpenAI.
  • AWS Bedrock.
• Tutorial for running experiments with multi-step LLM applications wit MongoDB and OpenAI.
• Adds Streamlit Cookbook for tracing model calls with collected user feedback on AI response.

​

Core Platform

• Enhanced filter functionality: Added the ability to edit filters and improved schema discovery within filters.
• Fixed pagination issue for events table.

​

Python SDK - Version [v0.2.44]

• Improved error tracking for the tracer: Enhanced the capture of error messages for custom-decorated functions.
• Git context enrichment: Added support for capturing Git branch status in traces and experiments.
• Introduced the disable_http_tracing parameter during tracer initialization to disable HTTP event tracing.
• Fixed the traceloop version to 0.30.0 to resolve protobuf dependency conflicts.

​

TypeScript SDK - Version [v1.0.33]

• Improved error tracking for the tracer: Enhanced the capture of error messages for traced functions.
• Git context enrichment: Added support for capturing Git branch status in traces and experiments.
• Introduced the disableHttpTracing parameter during tracer initialization to disable HTTP event tracing.

​

Documentation

• Standardized all JavaScript/TypeScript code examples to TypeScript across the documentation.
• Added troubleshooting guidance for SSL validation failures.
• Documented the disable_http_tracing/disableHttpTracing parameter in the SDK Reference.
• Removed references to init_from_session_id in favor of using init with the session_id parameter.
• Updated the Observability Tutorial Documentation/Cookbook to use enrichSession instead of setFeedback/setMetadata
• Integrations - added CrewAI Integration documentation.

​

Core Platform

​

Introducing Review Mode

• Tag traces with annotations from your Human Evaluators definitions
• Apply your custom criteria right in the UI
• Add comments when something interesting pops up

​

Python SDK - Version [v0.2.36]

• Reduced package size for AWS lambda usage
• Removed Langchain dependency. For using Langchain callbacks, install Langchain separately
• Add lambda, core, and eval poetry installation groups

​

TypeScript SDK - Version [v1.0.23]

• Reduced package size for AWS lambda usage
• Disabled CommonJS autotracing 3rd party packages: Anthropic, Bedrock, Pinecone, ChromaDB, Cohere, Langchain, LlamaIndex, OpenAI. Please use custom tracing for instrumenting Typescript.
• Refactor custom tracer for better initialization syntax and using typescript

​

Documentation

• Added Schema Overview documentation to describe our schemas in detail including a list of reserved properties.
• Added Client-side Evaluators documentation to describe the use of client-side evaluators for both tracing and experiments
• Updated Custom Spans documentation to add reference to tracing methods traceModel/traceTool/traceChain (TypeScript)
• Integrations - added LanceDB Integration documentation
• Integrations - added Zilliz Integration documentation