Skip to main content
Human annotation allows for manual review and evaluation of AI outputs by human reviewers.

​
Creating a Human Evaluator

  1. Navigate to the Evaluators tab in the HoneyHive console.
  2. Click Add Evaluator and select Human Evaluator.
humaneval

​
Evaluation Criteria

Define clear evaluation criteria for annotators in the Description field:

​
Configuration

​
Return Type

Options:
  1. Numeric: For ratings on a scale
  2. Binary: For yes/no evaluations
  3. Notes: For free-form text feedback
  4. Categorical: For categorization tasks

​
Rating Scale

For numeric return type and, in some cases, categorical return type (i.e where numerical labels are defined), specify the scale (e.g., 1-5).

​
Passing Range

Define the range of scores considered acceptable.

​
In-App Annotation

Once created, human evaluators are available throughout the UI - in traces, Review Mode or Annotation Queues. You can invite domain experts to annotate traces in any project.
humaneval