Using shared assets improves itteration

For LLMops, team members need to know what dataset was evaluated on which prompt and what metrics were used. This is simple in theory but without a principaled approach to managing assets, variantions arise and confusion follows. HoneyHive has created a simple, easy to use paradigm for managing the usage of commonly shared datasets, prompts, and metrics in the experimentation phase as well as in production.

In order to use custom honeyhive datasets, the dataset should already be uploaded into HoneyHive. The reccomended route is to upload your dataset through the gui. We also provide the ability to upload directly via our api.

Datasets

There are two options when it comes to datasets: public and custom. HoneyHive allows you to easily track the usage of both

import honeyhive as hh
from honeyhive.sdk.datasets import get_dataset

dataset = get_dataset("Email Writer Samples")

# Note: All datasets must be a list of dictionaries