Skip to main content
POST
Create an evaluator

Authorizations

Authorization
string
header
required

A fine-grained data plane API key (values begin with hh_fgdp_). Rooted at an organization, workspace, or project, it carries the permissions chosen when it was created, and only the operations whose paths begin with /v1/projects/{project_id}/ accept it. Create one in the HoneyHive app under the project's, workspace's, or organization's Settings → API Keys, on the Data Plane tab.

Path Parameters

project_id
string
required

Project that owns the evaluator; must be within the API key root scope.

Body

application/json
slug
string
required

Immutable, case-sensitive project-unique slug containing 1–64 ASCII letters, digits, underscores or hyphens. Must not be a valid ULID, regardless of case. Cannot change after creation.

Required string length: 1 - 64
Pattern: ^[a-zA-Z0-9_-]+$
definition
object
required

Evaluator definition with type python, llm, or human and the matching configuration object under that key.

message
string
default:""

Message for the initial version; defaults to an empty string.

publish
boolean
default:false

Publish the saved version atomically. Requires project.evaluator.publish in addition to the operation permission.

Response

Successful response

success
enum<boolean>
required
Available options:
true
data
object
required