Get API key

After signing up on the app, you can find your API key in the Settings page under Account.

Log user feedback and ground truth labels

If you have already logged a LLM request in HoneyHive, you can now log user feedback and any ground truth labels associated with that completion.

Using the generation_id that is returned, you can send arbitrary feedback to HoneyHive using the feedback endpoint.

We recommend three general categories of feedback:-

  1. Rating: Helps you log explicit ratings from your users on model performance. Examples include ”👍” / ”👎” or on a numeric scale (eg: “1-5”).
  2. Action: Helps you log implicit actions from your users. Examples include “Edited”, “Regenerated”, “Saved”, “Exited”, etc.
  3. Issues: Helps you log explicit issues pointed out by your users. Examples include “Inappropriate Content”, “Incorrect Answer”, etc.

Below is an example implementation from a writing-assistant app where we log ratings (thumbs_up, rating), user actions (corrected, saved) and a ground truth label (here, we use the end-user’s edited response as the ground truth label).

from honeyhive.sdk.tracer import HoneyHiveTracer
import honeyhive as hh

HONEYHIVE_API_KEY = ""

hh.api_key = HONEYHIVE_API_KEY

honeyhive_tracer = HoneyHiveTracer(
    project="New Project",
    name="Feedback Test",
)

# Replace your inference code here using with honeyhive_tracer

feedback = {
    "provided": True,
    "accepted": False,
}

# log feedback for the session
session_id = honeyhive_tracer.session_id
hh.sdk.feedback(session_id=session_id, feedback=feedback)

Logging user feedback for a particular event

In case you need to provide feedback for a specific step in your pipeline, you can do so by specifying the event_id as follows:

honeyhive.sessions.feedback(
    session_id = tracer.session_id,
    event_id = model_call.event_id,
    ground_truth = "INSERT_GROUND_TRUTH_LABEL",
    feedback = {
       "accepted": False
    }
)

Finish and view trace

Finally, we need to finish tracing the pipeline. This will send the trace to HoneyHive.

tracer.end_session()

The response will provide a link to the session in HoneyHive, where you can view the traces and user feedback.

You can now view this trace from within the HoneyHive platform by clicking on Datasets in the sidebar and then Traces. Trace