> ## Documentation Index
> Fetch the complete documentation index at: https://docs.honeyhive.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Use Portkey for LLM Evaluators

> Route LLM evaluator calls through Portkey's AI gateway to access any of its 1,600+ supported models.

[Portkey](https://portkey.ai/) is an AI gateway that provides a unified OpenAI-compatible API across 1,600+ models from providers including OpenAI, Anthropic, AWS Bedrock, Google, Mistral, and more. Because HoneyHive's LLM evaluators use an OpenAI-compatible interface, you can point them at Portkey's gateway to access any provider Portkey supports. This is useful when you need to:

* Run evaluators on models not natively supported in HoneyHive (AWS Bedrock, Mistral, Cohere, etc.)
* Apply Portkey **routing rules** (fallbacks, load balancing, budget limits) to evaluation calls
* Manage provider credentials centrally in Portkey rather than per-tool

## How it works

HoneyHive's LLM evaluators call a provider using two fields from the evaluator definition:

* `model_provider` - determines which credential set to use
* `model_name` - passed verbatim as the model identifier

Setting `model_provider` to `openai` and pointing the OpenAI `api_base` at Portkey's gateway redirects all evaluator calls through Portkey. The `model_name` field uses Portkey's `@provider-slug/model-id` format to tell Portkey which backend provider to target.

## Step 1: Configure provider keys

In HoneyHive, go to **Settings > Workspace > AI Providers** and click **OpenAI**. Enter:

| Field      | Value                       |
| ---------- | --------------------------- |
| `api_key`  | Your Portkey API key        |
| `api_base` | `https://api.portkey.ai/v1` |

Save. The OpenAI provider will now show as connected.

<Warning>
  Provider secrets are configured at the workspace level. Changing the OpenAI `api_base` to Portkey's gateway affects **all LLM evaluators** in the workspace that use the `openai` provider.

  To keep some evaluators on a direct OpenAI connection, use a separate workspace.
</Warning>

## Step 2: Set up a provider in Portkey's Model Catalog

In the [Portkey dashboard](https://app.portkey.ai/), add your provider credentials under **Model Catalog**:

1. Add credentials for the provider you want to use (e.g. Anthropic API key, AWS credentials)
2. Give it a slug, for example `anthropic-prod` or `bedrock-prod`
3. That slug becomes the `@provider-slug` prefix in your model names

See [Portkey's Model Catalog docs](https://portkey.ai/docs/product/ai-gateway/model-catalog) for supported providers and credential formats.

## Step 3: Update the evaluator's model name

The HoneyHive UI shows a fixed list of model names, so set the Portkey model name via API. Use a `PUT` request with just the fields to change - all other evaluator settings are preserved:

```bash theme={null}
curl -X PUT 'https://api.dp1.us.honeyhive.ai/v1/metrics/<metric_id>' \
  -H 'Authorization: Bearer <your-honeyhive-api-key>' \
  -H 'Content-Type: application/json' \
  -d '{
    "model_provider": "openai",
    "model_name": "@<provider-slug>/<model-id>"
  }'
```

Replace:

* `<metric_id>` - the ID of your LLM evaluator (visible in the evaluator URL in the HoneyHive console)
* `<your-honeyhive-api-key>` - your HoneyHive API key
* `<provider-slug>` - the slug you assigned in Portkey's Model Catalog
* `<model-id>` - the model ID in the format your target provider expects

<Warning>
  After setting the model via API, the evaluator editor will show "Model unavailable" for the Model field - this is expected. If you open the evaluator in the UI and save it, the dropdown will overwrite `model_name` with a listed value, reverting your Portkey configuration. Make all future changes to `model_provider` and `model_name` via the API.
</Warning>

### Example model names

| Target provider         | Example `model_name`                                        |
| ----------------------- | ----------------------------------------------------------- |
| OpenAI (via Portkey)    | `@openai-prod/gpt-4o`                                       |
| Anthropic (via Portkey) | `@anthropic-prod/claude-sonnet-4-6-20250929`                |
| Mistral (via Portkey)   | `@mistral-prod/mistral-large-latest`                        |
| AWS Bedrock - Anthropic | `@bedrock-prod/us.anthropic.claude-haiku-4-5-20251001-v1:0` |
| AWS Bedrock - Meta      | `@bedrock-prod/us.meta.llama3-3-70b-instruct-v1:0`          |

The model ID after the slug must match the format the target provider expects. Check the provider's own documentation for the correct model identifier string.

## Verify the configuration

Trigger an evaluator run by sending a trace that matches the evaluator's filters, or by running an experiment. If it succeeds, the evaluator result appears as normal. If it fails, check:

1. **Provider secret values** - confirm `api_key` and `api_base` are set correctly under Settings > AI Providers > OpenAI
2. **Portkey provider slug** - ensure the slug in `model_name` matches exactly what you configured in Portkey's Model Catalog
3. **Model ID format** - verify the model ID after the slug is valid for the target provider

<Note>
  Portkey logs all gateway requests in its dashboard. If HoneyHive's evaluator call reaches Portkey but the underlying provider returns an error, the Portkey logs will show the provider's error response.
</Note>

## Related

<CardGroup cols={2}>
  <Card title="LLM Evaluators" icon="robot" href="/v2/evaluators/llm">
    Create and configure LLM evaluators
  </Card>

  <Card title="Provider Keys" icon="key" href="/v2/workspace/provider-keys">
    Manage AI provider credentials in HoneyHive
  </Card>

  <Card title="Portkey Integration" icon="shuffle" href="/v2/integrations/portkey">
    Add HoneyHive tracing to Portkey-routed applications
  </Card>
</CardGroup>