> ## Documentation Index
> Fetch the complete documentation index at: https://langchain-5e9cc07a-preview-docsmd-1789592284-4ee7e30.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# API formats

> Use OpenAI Chat Completions, Anthropic Messages, or OpenAI Responses requests to call models across providers through the LLM Gateway.

<Note>
  The LLM Gateway is in [beta](/langsmith/release-stages).
</Note>

The standard LLM Gateway API supports three request and response formats. Choose the format your application already uses, then call bring-your-own-key or Gateway Credits models through the same endpoint.

## Compare API formats

| API format              | Base URL                                 | Prompt endpoint          | Compatible client                          |
| ----------------------- | ---------------------------------------- | ------------------------ | ------------------------------------------ |
| OpenAI Chat Completions | `https://gateway.smith.langchain.com/v1` | `POST /chat/completions` | OpenAI-compatible Chat Completions clients |
| Anthropic Messages      | `https://gateway.smith.langchain.com`    | `POST /v1/messages`      | Anthropic Messages clients                 |
| OpenAI Responses        | `https://gateway.smith.langchain.com/v1` | `POST /responses`        | OpenAI-compatible Responses clients        |

All formats authenticate with a workspace-scoped LangSmith API key. Pass it as the provider API key or as an `Authorization: Bearer` token.

These base URLs are for the US gateway on LangSmith Cloud. For other regions and for BYOC data planes, see [Check availability](/langsmith/llm-gateway-how-it-works#check-availability).

For bring-your-own-key models, set `model` to `<provider>/<model>`, such as `openai/gpt-5.4-mini`, `anthropic/claude-opus-5`, or `azure/<deployment-name>`. For Gateway Credits models, pass a supported model name, such as `moonshotai/kimi-k3`.

## Use Chat Completions

Point an OpenAI-compatible client at `https://gateway.smith.langchain.com/v1`. For the full request and response schema, see the [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat).

<CodeGroup>
  ```bash cURL theme={null}
  curl https://gateway.smith.langchain.com/v1/chat/completions \
      -H "Authorization: Bearer $LANGSMITH_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"model":"anthropic/claude-opus-5","messages":[{"role":"user","content":"Hello!"}]}'
  ```

  ```python Python theme={null}
  import os

  from openai import OpenAI

  client = OpenAI(
      base_url="https://gateway.smith.langchain.com/v1",
      api_key=os.environ["LANGSMITH_API_KEY"],
  )
  response = client.chat.completions.create(
      model="anthropic/claude-opus-5",
      messages=[{"role": "user", "content": "Hello!"}],
  )
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://gateway.smith.langchain.com/v1",
    apiKey: process.env.LANGSMITH_API_KEY,
  });
  const response = await client.chat.completions.create({
    model: "anthropic/claude-opus-5",
    messages: [{ role: "user", content: "Hello!" }],
  });
  ```
</CodeGroup>

## Use Messages

Point an Anthropic client at `https://gateway.smith.langchain.com`. For the full request and response schema, see the [Anthropic Messages API](https://docs.anthropic.com/en/api/messages).

<CodeGroup>
  ```bash cURL theme={null}
  curl https://gateway.smith.langchain.com/v1/messages \
      -H "Authorization: Bearer $LANGSMITH_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"model":"openai/gpt-5.4-mini","max_tokens":1024,"messages":[{"role":"user","content":"Hello!"}]}'
  ```

  ```python Python theme={null}
  import os

  import anthropic

  client = anthropic.Anthropic(
      base_url="https://gateway.smith.langchain.com",
      api_key=os.environ["LANGSMITH_API_KEY"],
  )
  message = client.messages.create(
      model="openai/gpt-5.4-mini",
      max_tokens=1024,
      messages=[{"role": "user", "content": "Hello!"}],
  )
  ```

  ```typescript TypeScript theme={null}
  import Anthropic from "@anthropic-ai/sdk";

  const client = new Anthropic({
    baseURL: "https://gateway.smith.langchain.com",
    apiKey: process.env.LANGSMITH_API_KEY,
  });
  const message = await client.messages.create({
    model: "openai/gpt-5.4-mini",
    max_tokens: 1024,
    messages: [{ role: "user", content: "Hello!" }],
  });
  ```
</CodeGroup>

## Use Responses

Point an OpenAI-compatible client at `https://gateway.smith.langchain.com/v1`. For the full request and response schema, see the [OpenAI Responses API](https://platform.openai.com/docs/api-reference/responses).

<CodeGroup>
  ```bash cURL theme={null}
  curl https://gateway.smith.langchain.com/v1/responses \
      -H "Authorization: Bearer $LANGSMITH_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"model":"anthropic/claude-opus-5","input":"Hello!"}'
  ```

  ```python Python theme={null}
  import os

  from openai import OpenAI

  client = OpenAI(
      base_url="https://gateway.smith.langchain.com/v1",
      api_key=os.environ["LANGSMITH_API_KEY"],
  )
  response = client.responses.create(
      model="anthropic/claude-opus-5",
      input="Hello!",
  )
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://gateway.smith.langchain.com/v1",
    apiKey: process.env.LANGSMITH_API_KEY,
  });
  const response = await client.responses.create({
    model: "anthropic/claude-opus-5",
    input: "Hello!",
  });
  ```
</CodeGroup>

## Enable prompt caching

OpenAI models (Chat Completions and Responses) support implicit prompt caching automatically, no extra parameters are required.

Anthropic models and some older OpenAI models require explicit opt-in to prompt caching. Pass provider-specific fields in your request body when calling these models through any standard gateway endpoint.

<Note>
  Explicit caching support is a temporary measure while a gateway-level caching policy is being developed. The following fields are passed through to the upstream provider.
</Note>

### Anthropic models

Include `prompt_cache_options` with a `ttl` value:

<CodeGroup>
  ```bash cURL theme={null}
  curl https://gateway.smith.langchain.com/v1/responses \
      -H "Authorization: Bearer $LANGSMITH_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "model": "anthropic/claude-opus-5",
        "input": "Hello!",
        "prompt_cache_options": {"ttl": "30m"}
      }'
  ```

  ```python Python theme={null}
  import os

  from openai import OpenAI

  client = OpenAI(
      base_url="https://gateway.smith.langchain.com/v1",
      api_key=os.environ["LANGSMITH_API_KEY"],
  )
  response = client.responses.create(
      model="anthropic/claude-opus-5",
      input="Hello!",
      extra_body={"prompt_cache_options": {"ttl": "30m"}},
  )
  ```

  ```typescript TypeScript theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://gateway.smith.langchain.com/v1",
    apiKey: process.env.LANGSMITH_API_KEY,
  });
  const response = await client.responses.create({
    model: "anthropic/claude-opus-5",
    input: "Hello!",
    // @ts-ignore — provider-specific field
    prompt_cache_options: { ttl: "30m" },
  });
  ```
</CodeGroup>

The same field works with the Chat Completions endpoint:

```bash cURL theme={null}
curl https://gateway.smith.langchain.com/v1/chat/completions \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "anthropic/claude-opus-5",
      "messages": [{"role": "user", "content": "Hello!"}],
      "prompt_cache_options": {"ttl": "30m"}
    }'
```

### Older OpenAI models

Some older OpenAI models support explicit cache control via `prompt_cache_retention`. Set it to `"in_memory"` for most models. For `gpt-5.5` specifically, use `"24h"`:

```bash cURL (most older models) theme={null}
curl https://gateway.smith.langchain.com/v1/responses \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "openai/gpt-5.4-mini",
      "input": "Hello!",
      "prompt_cache_retention": "in_memory"
    }'
```

```bash cURL (gpt-5.5 specifically) theme={null}
curl https://gateway.smith.langchain.com/v1/responses \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "openai/gpt-5.5",
      "input": "Hello!",
      "prompt_cache_retention": "24h"
    }'
```

For full `prompt_cache_retention` documentation, see the [OpenAI prompt caching guide](https://developers.openai.com/api/docs/guides/prompt-caching#prompt-cache-retention).

## Understand translation behavior

The endpoint determines the format your application sends and receives. The model ID determines the upstream provider.

* When the provider supports the selected format natively, the gateway preserves that format.
* Otherwise, the gateway translates the request into a format supported by the provider and translates the response back, including streaming responses.
* Translation can reject fields that cannot be represented in the target provider format. Use [Direct model access](/langsmith/llm-gateway-direct-model-access) when provider-native behavior is required.

Every request resolves the same Provider Secrets, policies, and tracing configuration regardless of format.

## List models

Call `GET /v1/models` to list models available from providers configured for the workspace and from [Gateway Credits](/langsmith/llm-gateway-credits). The gateway returns a single OpenAI-compatible list:

```bash theme={null}
curl https://gateway.smith.langchain.com/v1/models \
    -H "Authorization: Bearer $LANGSMITH_API_KEY"
```

```json theme={null}
{
  "object": "list",
  "data": [
    {"id": "openai/gpt-5.4-mini", "object": "model"},
    {"id": "fireworks/accounts/fireworks/models/glm-5p2", "object": "model"},
    {"id": "anthropic/claude-opus-5", "object": "model"},
    {"id": "moonshotai/kimi-k3", "object": "model"}
  ]
}
```

Bring-your-own-key model IDs use the form `<provider>/<model>`. Hosted models use the slug shown in the response. Pass either ID exactly as shown when making a call. A bring-your-own-key provider without a configured secret is omitted; hosted models do not require a provider secret.

## Handle errors

| Status or symptom                                           | Meaning                                                                                                              |
| ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| `400 Bad Request`                                           | The request is malformed, the model ID is unavailable or incorrectly formatted, or the request cannot be translated. |
| `401 Unauthorized`                                          | The LangSmith API key is missing or invalid.                                                                         |
| `403 Forbidden`                                             | The key does not have the required gateway permissions.                                                              |
| `429 Too Many Requests`                                     | A gateway rate limit or an upstream provider rate limit was reached.                                                 |
| No models with a provider prefix appear in `GET /v1/models` | The provider may not be configured or may not have returned a model catalog.                                         |

For setup-specific resolutions, see the [Quickstart](/langsmith/llm-gateway-quickstart).

## See also

* [Quickstart](/langsmith/llm-gateway-quickstart): make your first request and view its trace.
* [How the gateway works](/langsmith/llm-gateway-how-it-works): what happens to each request, and which hostname to use in each region and on BYOC.
* [Direct model access](/langsmith/llm-gateway-direct-model-access): bypass format translation and use provider-native APIs.
* [Model fallbacks](/langsmith/llm-gateway-fallbacks): retry requests against backup models.

***

<div className="source-links">
  <Callout icon="terminal-2">
    [Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
  </Callout>

  <Callout icon="edit">
    [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/langsmith/llm-gateway-api-formats.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
  </Callout>
</div>