Anthropic-compatible API

TurboLLM speaks the Anthropic Messages API. Point any Anthropic SDK client — or Claude Code itself — at your local gateway and it runs against your own model, no cloud round-trip.

Endpoint

TurboLLM exposes a single Messages endpoint alongside its OpenAI-compatible surface:

POST http://localhost:6996/v1/messages

The gateway loads the requested model on demand, so the first request after a cold start may take a moment while the model spins up.

What's supported

Base URL & auth

The base URL is http://localhost:6996. This is exactly what makes Claude Code work locally — set ANTHROPIC_BASE_URL to it.

Anthropic-style clients send an auth token. With Require API key turned off, any non-empty token is accepted. With it turned on, use your TurboLLM API key as the token.

Request example

A standard Messages request takes model, max_tokens, and a messages array.

curl http://localhost:6996/v1/messages \
  -H "content-type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -H "x-api-key: $TURBOLLM_API_KEY" \
  -d '{
    "model": "your-local-model",
    "max_tokens": 1024,
    "messages": [
      { "role": "user", "content": "Hello from TurboLLM" }
    ]
  }'

from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:6996",
    api_key="your-turbollm-key",  # any non-empty token if key auth is off
)

message = client.messages.create(
    model="your-local-model",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello from TurboLLM"},
    ],
)

print(message.content)