Messages API (/messages)

Learn how to use Messages API through TrueFoundry’s AI Gateway for interacting with Claude models.

API Reference: POST /messages

Provider capabilities

The table below summarizes gateway support for this endpoint by provider.

Legend:

• ✅ Supported by provider and TrueFoundry
• Provided by provider, but not by TrueFoundry
• Provider does not support this feature

Provider	Messages API
Anthropic	✅

For every gateway endpoint and provider, see Supported APIs.

Anthropic’s Messages API is a powerful interface for interacting with Claude models. When using TruefFundry as your model gateway, you can access this API through a proxy endpoint that handles authentication and routing to the appropriate model.

Prerequisites

To use the Anthropic Messages API through TrueFoundry, you’ll need:

1. TrueFoundry API Key
2. Model account configured in TrueFoundry (Anthropic)
3. Python environment with anthropic sdk library installed

Using the Anthropic SDK

The Anthropic Python SDK provides a convenient way to interact with Claude models. Here’s how to configure it to work with the TrueFoundry proxy:

The AI Gateway accepts both Anthropic SDK auth patterns and translates internally:

• api_key=TFY_API_KEY - SDK sends the x-api-key header
• auth_token=TFY_API_KEY — SDK sends the Authorization: Bearer header

Either works; the request body is identical. api_key is the idiomatic Anthropic SDK pattern - use it unless you have a reason to send a Bearer token.

from anthropic import Anthropic

BASE_URL = "{GATEWAY_BASE_URL}"
API_KEY = "your-truefoundry-api-key"

# Configure the Anthropic client to use TrueFoundry's Gateway
client = Anthropic(
    api_key=API_KEY,
    base_url=BASE_URL
)

# Make a request to the Messages API
def generate_response():
    response = client.messages.create(
        model="anthropic/claude-3-5",  # The model name configured in TrueFoundry
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": "Hello, Claude! Please explain quantum computing in simple terms."
            }
        ]
    )

    print(response.content)

generate_response()

Request Format

When using the Messages API through TrueFoundry, your request should follow this format:

{
  "model": "tfy-model-name",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": "Hello, Claude! Please explain quantum computing in simple terms."
    }
  ]
}

Response Format

The response from the Messages API will have this structure:

{
  "id": "msg_01XB89YSAA2VGMCF3ZS8ATTA1B",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Quantum computing is like traditional computing but it uses quantum bits or 'qubits' instead of regular bits. While traditional bits can only be in a state of 0 or 1, qubits can exist in multiple states simultaneously thanks to a quantum property called 'superposition.' This allows quantum computers to process certain types of information much faster than regular computers.\n\nAnother key quantum property is 'entanglement,' where qubits become connected and the state of one instantly affects the other, no matter the distance between them.\n\nThese properties give quantum computers the potential to solve certain complex problems much faster than traditional computers, like factoring large numbers (important for encryption) or simulating molecular structures (useful for drug development).\n\nHowever, quantum computers are still in early development stages. They're extremely sensitive to their environment and require special conditions like ultra-cold temperatures to operate. They're not replacements for regular computers but specialized tools for specific types of problems."
    }
  ],
  "model": "claude-3-opus-20240229",
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 14,
    "output_tokens": 178
  }
}

Prompt Caching

Prompt caching reuses previously computed prompt prefixes (a large system prompt, a shared context document, or a set of tool definitions) so the provider can skip reprocessing the cached portion on subsequent requests. This lowers both latency and cost.

There are three ways to cache with Claude models, and you can combine them:

• Explicit caching — you mark cache breakpoints yourself by adding cache_control to individual blocks (a system block, a message content block, or a tool definition). See Anthropic’s prompt caching guide.
• Automatic caching — you add a single cache_control field at the top level of the request body, and the breakpoint is placed at the end of the prompt for you, so you don’t manage breakpoints yourself.
• Header-based caching — you send the x-tfy-cache-control header and leave the body untouched. Equivalent to automatic caching, for clients that can’t edit the request body. See Caching with a header.

The AI Gateway forwards cache_control to native Claude providers unchanged — it does not strip it (the request is already in Anthropic’s format). The AI Gateway only strips cache_control for providers that don’t accept it, such as OpenAI or Gemini, which aren’t served natively on the Messages endpoint (see the note below).

Provider support

The Messages endpoint is served natively (in Anthropic’s format) by Anthropic and by Claude models on Bedrock, Google Vertex, and Azure AI Foundry. All three ways to cache work on every one of them:

Provider	Explicit (block-level)	Automatic (top-level cache_control or x-tfy-cache-control)
Anthropic (direct)	✅	✅
Claude Platform on AWS	✅	✅
Azure AI Foundry (Claude)	✅	✅
AWS Bedrock (Claude)	✅	✅
Google Vertex (Claude)	✅	✅

Only Anthropic, Claude Platform on AWS, and Azure AI Foundry read a top-level cache_control themselves — Bedrock and Google Vertex reject it. The AI Gateway papers over that difference by relocating a top-level cache_control onto the last cacheable block before forwarding, which every native Claude provider honours. The end result is the same either way: one breakpoint at the end of the prompt, caching the whole prefix.

On Bedrock, prompt caching is enabled by default when the request goes through the InvokeModel API (which the Messages endpoint uses for Claude models). You can set explicit cache checkpoints at any point in your request body — across system blocks, message content blocks, and tool definitions — by attaching cache_control to each block you want to mark. Unlike the Converse-based /chat/completions path, these are forwarded as native Anthropic cache_control rather than rewritten into cachePoint markers.

Non-Claude models (and provider-managed providers such as OpenAI or Gemini) are served on the Messages endpoint by translating the request into Chat Completions format. In that translation cache_control is dropped, but the provider’s own prefix caching still applies transparently, and any cached-token counts are still reported in usage.

Explicit caching

Add "cache_control": {"type": "ephemeral"} to any system block, message content block, or tool definition you want cached. This works on every native Claude provider:

cURL

curl -X POST "{GATEWAY_BASE_URL}/messages" \
  -H "Authorization: Bearer $TFY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic-main/claude-sonnet-4-20250514",
    "max_tokens": 1024,
    "system": [
      {
        "type": "text",
        "text": "<LONG_SYSTEM_PROMPT_OVER_THE_MIN_TOKENS>",
        "cache_control": { "type": "ephemeral" }
      }
    ],
    "messages": [
      { "role": "user", "content": "Summarize the key rules from the system prompt." }
    ]
  }'

Automatic caching

Add a single cache_control field at the top level of the request body instead of marking individual blocks. The breakpoint lands on the last cacheable block and advances as the conversation grows — useful for multi-turn chats. This works on every native Claude provider:

cURL

curl -X POST "{GATEWAY_BASE_URL}/messages" \
  -H "Authorization: Bearer $TFY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic-main/claude-opus-4-8",
    "max_tokens": 1024,
    "cache_control": { "type": "ephemeral" },
    "system": "You are a helpful assistant that remembers our conversation.",
    "messages": [
      { "role": "user", "content": "My name is Alex. I work on machine learning." },
      { "role": "assistant", "content": "Nice to meet you, Alex!" },
      { "role": "user", "content": "What did I say I work on?" }
    ]
  }'

Caching with a header

When the request body is built by a framework or an off-the-shelf agent you can’t modify, send the x-tfy-cache-control header instead. The AI Gateway injects the breakpoint at the end of the prompt for you, so the whole prefix — tool definitions, system prompt, and earlier turns — is cached:

cURL

curl -X POST "{GATEWAY_BASE_URL}/messages" \
  -H "Authorization: Bearer $TFY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-tfy-cache-control: {\"type\": \"ephemeral\"}" \
  -d '{
    "model": "anthropic-main/claude-sonnet-4-20250514",
    "max_tokens": 1024,
    "system": "<LONG_SYSTEM_PROMPT_OVER_THE_MIN_TOKENS>",
    "messages": [
      { "role": "user", "content": "Summarize the key rules from the system prompt." }
    ]
  }'

The header value is a JSON object with the same shape as the cache_control field you would have written in the body:

Field	Required	Description
type	Yes	Cache type forwarded to the provider. Claude models accept ephemeral.
ttl	No	Cache duration in Anthropic’s format — 5m, 1h, 2d. Ignored by providers that don’t accept a duration.

A cache_control in the body always wins — if the request already marks any block, the header is ignored, so hand-placed breakpoints are never overwritten. The value must be valid JSON and, if you set ttl, it must match the 5m / 1h / 2d format, otherwise the AI Gateway rejects the request with a 400.

Don’t confuse x-tfy-cache-control with x-tfy-cache-config, which turns on the AI Gateway’s own exact-match and semantic response cache. They work at different layers and can be used together.

Anthropic enforces a minimum cacheable prefix length; shorter prompts accept the cache_control hint but are not actually cached:

Minimum tokens	Models
4096	Claude Mythos Preview, Opus 4.7, Opus 4.6, Opus 4.5, Haiku 4.5
2048	Sonnet 4.6, Haiku 3.5, Haiku 3
1024	Sonnet 4.5, Opus 4.1, Opus 4, Sonnet 4, Sonnet 3.7

Cache usage in the response

When caching is active, the response usage object reports the cached token counts:

{
  "usage": {
    "input_tokens": 120,
    "output_tokens": 210,
    "cache_creation_input_tokens": 4096,
    "cache_read_input_tokens": 0
  }
}

• cache_creation_input_tokens: tokens written to the cache (first call).
• cache_read_input_tokens: tokens served from the cache (subsequent calls).

Advanced Features

The Messages API supports several advanced features:

System Prompts

You can include a system prompt to guide Claude’s behavior:

client.messages.create(
    model="anthropic/claude-3-5",
    system="You are a helpful AI assistant that specializes in explaining complex topics simply.",
    messages=[
        {"role": "user", "content": "Explain quantum entanglement."}
    ]
)

Multi-turn Conversations

For multi-turn conversations, include previous messages:

client.messages.create(
    model="anthropic/claude-3-5",
    messages=[
        {"role": "user", "content": "What is machine learning?"},
        {"role": "assistant", "content": "Machine learning is a subset of artificial intelligence..."},
        {"role": "user", "content": "Can you explain supervised vs unsupervised learning?"}
    ]
)

Streaming Responses

For streaming responses, use the streaming parameter:

with client.messages.stream(
    model="anthropic/claude-3-5",
    messages=[{"role": "user", "content": "Write a short poem about AI."}]
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

    # Access the final message at the end
    print("\nFinal message:", stream.get_final_message())