LLM Gateway quickstart

The LLM Gateway is in beta.

The LLM Gateway calls models across configured providers through one endpoint with one LangSmith API key. Send a request, view its trace, then set a spend limit.

An administrator must enable the gateway, add a provider secret, and grant access once for your workspace. After that, you need only a workspace-scoped LangSmith API key attached to a role with the gateway:invoke and workspaces:read permissions.

Send a request

A gateway call is an ordinary model request pointed at the gateway base URL and authenticated with your LangSmith API key. Use Chat completions to call the gateway from an application you already have, or Deep Agents to build an agent that routes through it.

Chat completions
Deep Agents

Point any OpenAI-compatible client at https://gateway.smith.langchain.com/v1 and set model to a provider-prefixed ID.

export LANGSMITH_API_KEY="lsv2_..."

curl https://gateway.smith.langchain.com/v1/chat/completions \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model":"anthropic/claude-opus-5","messages":[{"role":"user","content":"Explain what an LLM gateway does in one sentence."}]}'

import os

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.smith.langchain.com/v1",
    api_key=os.environ["LANGSMITH_API_KEY"],
)
response = client.chat.completions.create(
    model="anthropic/claude-opus-5",
    messages=[{"role": "user", "content": "Explain what an LLM gateway does in one sentence."}],
)
print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.smith.langchain.com/v1",
  apiKey: process.env.LANGSMITH_API_KEY,
});
const response = await client.chat.completions.create({
  model: "anthropic/claude-opus-5",
  messages: [{ role: "user", content: "Explain what an LLM gateway does in one sentence." }],
});
console.log(response.choices[0].message.content);

The same endpoint also accepts Anthropic Messages and OpenAI Responses requests. See API formats.

Set LANGSMITH_GATEWAY to route every supported chat model in the process through the gateway.

# pip install "deepagents>=0.1.0"
# export LANGSMITH_API_KEY="lsv2_..."
# export LANGSMITH_GATEWAY="true"
from deepagents import create_deep_agent

agent = create_deep_agent(model="anthropic:claude-opus-5")
result = agent.invoke({"messages": [{"role": "user", "content": "Explain what an LLM gateway does in one sentence."}]})
print(result["messages"][-1].content)

// npm install deepagents
// export LANGSMITH_API_KEY="lsv2_..."
// export LANGSMITH_GATEWAY="true"
import { createDeepAgent } from "deepagents";

const agent = createDeepAgent({ model: "anthropic:claude-opus-5" });
const result = await agent.invoke({
  messages: [{ role: "user", content: "Explain what an LLM gateway does in one sentence." }],
});
console.log(result.messages[result.messages.length - 1].content);

To route specific calls rather than all calls, use the standard endpoint in the Chat completions tab and set model to a provider-prefixed ID.

View the trace

Open LangSmith and go to the tracing project named gateway or gateway-<short_api_key>-<api_key_id> in your workspace. Your request appears there with its token counts, cost, and latency.

Set a spend limit

Go to LLM Gateway in LangSmith and create a spend policy, such as a daily $10 cap on your API key. Once the cap is reached, the gateway returns a 402 with a message naming the policy that blocked the request:

{"error": "Request blocked by gateway policies: R&D Spend Cap"}

For the full guide, see Spend policies.

These examples use the US gateway. For the EU, APAC, and AWS hostnames, see Use a regional gateway. For BYOC, see Use a BYOC data plane.

Next steps

Overview: what the gateway provides, how credentials are managed, and when to use the standard API.
How the gateway works: what happens to each request, how credentials resolve, and where the gateway is available.
API formats: use Chat Completions, Messages, or Responses through the standard endpoint.
Set up coding agents: route Claude Code, Codex, Gemini CLI, or Deep Agents Code through the gateway.
Direct model access: use provider-native request and response formats.
Prompt Hub with the gateway: route Prompt Hub model calls through the gateway.
Data policy: prevent sensitive data from reaching providers.

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.

Core capabilities

Administration and governance

Advanced

Next steps

​Next steps

Next steps