Concepts

What is User Feedback?

Any feedback that the end-user provides about their experience is considered user feedback. This can be of two types:

  1. Explicit Feedback: Any feedback provided by the user explicitly in your app. Examples include 👍/👎, ratings (1-n), etc.
  2. Implicit Feedback: Any actions within your app’s UI by your user that may indicate user experience and intent. Examples include clicking Copy, Regenerate, etc.
User Feedback vs Human Evaluator: User feedback field is used to track feedback provided by your end-users. Feedback from domain experts or other internal team members is considered a human evaluator in HoneyHive and is tracked on the metrics field.

Return Types

We accept all primary return types as user feedback. This includes:

Return TypeAvailable MeasurementsNotes
BooleanPercentage True/False---
NumberSum, Avg, Median, Min, Max, P95, P98, P99---
StringUsed for filters and group by

For complex data types, we allow you to drill down to the nested fields or specific positions in the array.

So, for example, if you pass feedback like:

{
  "rating": 5,
  "comment": "Great experience",
  "step1": {
    "retry": true
  },
  "edits": [
    {
      "value": "New York"
    }
  ]
}

You can chart feedback.step1.retry as a boolean field or feedback.edits.0.value as a string field.

Nesting Limitations: For complex data types like objects and arrays, we allow max 5 levels of nested objects and max 2 levels of nested arrays.

Reserved Fields

The following fields are reserved for user feedback:

ground_truth - The ground truth value for the output of a trace or span. This is used to compare the actual output with the expected output. It is rendered differently in the UI.

Instructions

Defining on Trace

You can pass feedback on your trace using the below function.

In case your LLM execution is split across a backend and frontend, then refer to the distributed tracing docs to make sure the feedback is set to the correct trace.

Prerequisites: You have already set tracing for your code as described here.

Expected Time: 1-2 minutes

What’s next