Run experiments using datasets stored and managed in HoneyHive UI.
Sample script for running experiments when managing datasets in HoneyHive Cloud
Create your dataset in jsonl format
market_dataset.jsonl
and paste the following content:Upload your dataset to HoneyHive
Create the flow you want to evaluate
inputs
and ground_truths
fields as defined in your dataset will be passed to this function.
For example, in one execution of this function, inputs
might contain a dictionary like:ground_truths
might contain a dictionary like:outputs
field of each run in the experiment and will be accessible to your evaluator function, as we will see below.(Optional) Setup Evaluators
inputs
and ground_truths
, the evaluator function has access to the return value from function_to_evaluate
, which is mapped to outputs
. In this example, outputs
would contain a string with the model response, such as:Run experiment
evaluate
: