> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeyhive.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Import from Hugging Face

> How to import datasets from HuggingFace Datasets to HoneyHive.

Since HoneyHive's datasets don't follow a fixed schema format, we have an automatic integration with HuggingFace datasets (or any kind of dataset management tool) to import datasets into HoneyHive.

## Upload a dataset through the SDK

On a high level, all we need to do is

* define our mapping of inputs-outputs
* importing batch size to setup the integration.

<Note>We recommend importing the data in batches of 100 rows at a time.</Note>

**Prerequisites**

* You have already created a project in HoneyHive, as explained [here](/workspace/projects).
* You have an API key for your project, as explained [here](/sdk-reference/authentication).

**Expected time:** few minutes

<Steps>
  <Step title="Installation">
    To install our SDK, run the following commands in the shell.

    <CodeGroup>
      ```shell Python theme={null}
      pip install honeyhive datasets
      ```
    </CodeGroup>
  </Step>

  <Step title="Authentication & Imports">
    To authenticate your SDK, you need to pass your API key.

    <CodeGroup>
      ```python Python theme={null}
      import honeyhive
      from honeyhive.models import components, operations
      from datasets import load_dataset

      hhai = honeyhive.HoneyHive(
        bearer_auth='YOUR_API_KEY',
        server_url='HONEYHIVE_SERVER_URL' # Optional / Required for self-hosted or dedicated deployments
      )
      ```
    </CodeGroup>
  </Step>

  <Step title="Create the HoneyHive dataset">
    Give your new dataset a name and pass the project name to which you want to associate the dataset.

    Keep the generated `dataset_id` handy for future reference.

    <CodeGroup>
      ```python Python theme={null}
      eval_dataset = hhai.datasets.create_dataset(request=components.CreateDatasetRequest(
        project='YOUR_PROJECT_NAME',
        name='DATASET_NAME',
      ))

      dataset_id = eval_dataset.object.result.inserted_id
      ```
    </CodeGroup>
  </Step>

  <Step title="Pass your data in batches with a mapping">
    Now, using the `dataset_id`, you can pass your data list and provide a mapping to the fields.

    We'll create unique datapoints for each entry in the JSON list. The `datapoint_id` on those entries will be used for joining traces in experiment runs in the future.

    <Note>Any field not defined in the mapping is set on the `metadata` of the datapoint.</Note>

    <CodeGroup>
      ```python Python theme={null}
      dataset = load_dataset("lhoestq/demo1") 
      dataset = list(dataset['train'])  # turn the dataset into a list of dictionaries
      datapoint_ids = []

      for i in range(0, len(dataset), 100):
          dataset_request = operations.AddDatapointsRequestBody(
              project = 'YOUR_PROJECT_NAME',
              data = dataset[i:i+100], # list of dictionaries
              mapping = operations.Mapping(
                  inputs=[
                    'review', # input fields
                  ],
                  ground_truth=[],
                  history=[]
              ),
          )

          datapoints = hhai.datasets.add_datapoints(
              dataset_id = dataset_id, # dataset_id from the previous step
              request_body = dataset_request
          )

          datapoint_ids.append(datapoints.object.datapoint_ids)
      ```
    </CodeGroup>
  </Step>
</Steps>

You have successfully uploaded your HuggingFace dataset to HoneyHive using the SDK.

You can now view your dataset in the HoneyHive UI.

## Next steps

<CardGroup cols={1}>
  <Card title="Running experiments" icon="table" href="/evaluation/quickstart">
    Learn how to run experiments on your dataset.
  </Card>
</CardGroup>