> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Responses API (/responses)
> Learn how to use OpenAI's Responses API through TrueFoundry AI Gateway for creating, retrieving, and managing text and multimodal completions.
**API Reference:** [`POST /responses`](/docs/api-reference/responses/model-responses)
## Provider capabilities
The table below summarizes gateway support for this endpoint by provider.
Legend:
* **✅** Supported by provider and TrueFoundry
* Provided by provider, but not by TrueFoundry
* Provider does not support this feature
| Provider | Model Response |
| ------------ | ----------------------------------------------------------- |
| OpenAI | ✅ |
| Azure OpenAI | ✅ |
| Anthropic | |
| Bedrock | |
| Vertex | |
| Cohere | |
| Gemini | |
| Groq | |
| Cerebras | |
| Together-AI | |
| xAI | |
| DeepInfra | |
| OpenRouter | ✅ |
| Databricks | ✅ |
For every gateway endpoint and provider, see [Supported APIs](/docs/ai-gateway/intro-to-llm-gateway#supported-apis).
This guide explains how to use the OpenAI client to interact with TrueFoundry's responses endpoint for making inference requests. Any provider marked supported above serves this endpoint natively — the request is forwarded as-is rather than translated into a chat completion.
On **Databricks**, native support covers the `databricks-gpt-5*` foundation model endpoints. Other Databricks endpoints accept `/responses` but are translated into a chat completion — see [Databricks Responses API](/docs/ai-gateway/databricks-models#responses-api).
The AI Gateway also uses the Responses API in the opposite direction, serving some Chat Completions requests through it upstream — see [Chat completions served through the Responses API](#chat-completions-served-through-the-responses-api) below.
## Authentication
You'll need a TrueFoundry API key to authenticate your requests. You can find authentication details from [here](/docs/ai-gateway/authentication).
You'll also need to set the `x-tfy-provider-name` header to the name of the provider integration you're using.
For example, if you're using a provider integration named `my-openai-provider`, you'll set the `x-tfy-provider-name` header to `my-openai-provider`.
```python lines theme={"dark"}
from openai import OpenAI
client = OpenAI(
base_url="{GATEWAY_BASE_URL}",
api_key="your_truefoundry_api_key",
default_headers={"x-tfy-provider-name": "my-openai-provider"}
)
```
## Text Completion
To get a text completion response:
```python lines theme={"dark"}
response = client.responses.create(
model="your_truefoundry_model_name", // tfy model name of a provider that supports /responses
input=[{"role": "user", "content": "Your prompt here"}]
)
print(response)
```
### Image Inputs
For tasks involving images:
```python lines theme={"dark"}
response = client.responses.create(
model="your_truefoundry_model_name", // tfy model name of a provider that supports /responses
input=[
{"role": "user", "content": "Your prompt here"},
{
"role": "user",
"content": [
{
"type": "input_image",
"image_url": "your_image_url"
}
]
}
]
)
response_id = response.id
```
### Exprected Response
```json lines theme={"dark"}
{
"id": "resp_6847fa2670f8819887e2d14c08bdc5a305d8dc9b22b6f0dd",
"created_at": 1749547558,
"error": null,
"incomplete_details": null,
"instructions": null,
"metadata": {},
"model": "gpt-4o-mini-2024-07-18",
"object": "response",
"output": [
{
"id": "msg_6847fa35e8c48198a5120ee3d38c353105d8dc9b22b6f0dd",
"content": [
{
"annotations": [],
"text": "It seems you might be looking to start a discussion or ask a question! How can I assist you today?",
"type": "output_text"
}
],
"role": "assistant",
"status": "completed",
"type": "message"
}
],
"parallel_tool_calls": true,
"temperature": 1,
"tool_choice": "auto",
"tools": [],
"top_p": 1,
"max_output_tokens": null,
"previous_response_id": null,
"reasoning": {
"effort": null,
"generate_summary": null,
"summary": null
},
"status": "completed",
"text": {
"format": {
"type": "text"
}
},
"truncation": "disabled",
"usage": {
"input_tokens": 10,
"input_tokens_details": {
"cached_tokens": 0
},
"output_tokens": 23,
"output_tokens_details": {
"reasoning_tokens": 0
},
"total_tokens": 33
},
"user": null,
"background": false,
"service_tier": "default",
"store": true,
"provider": "openai"
}
```
## Managing Responses
### Retrieve Response
Retrieve a response by id.
```python lines theme={"dark"}
response = client.responses.retrieve(
response_id=response_id
)
print(response)
```
### Expected Output
```json lines theme={"dark"}
{
"id": "resp_6847fa2670f8819887e2d14c08bdc5a305d8dc9b22b6f0dd",
"created_at": 1749547558,
"error": null,
"incomplete_details": null,
"instructions": null,
"metadata": {},
"model": "gpt-4o-mini-2024-07-18",
"object": "response",
"output": [
{
"id": "msg_6847fa35e8c48198a5120ee3d38c353105d8dc9b22b6f0dd",
"content": [
{
"annotations": [],
"text": "It seems you might be looking to start a discussion or ask a question! How can I assist you today?",
"type": "output_text"
}
],
"role": "assistant",
"status": "completed",
"type": "message"
}
],
"parallel_tool_calls": true,
"temperature": 1,
"tool_choice": "auto",
"tools": [],
"top_p": 1,
"max_output_tokens": null,
"previous_response_id": null,
"reasoning": {
"effort": null,
"generate_summary": null,
"summary": null
},
"status": "completed",
"text": {
"format": {
"type": "text"
}
},
"truncation": "disabled",
"usage": {
"input_tokens": 10,
"input_tokens_details": {
"cached_tokens": 0
},
"output_tokens": 23,
"output_tokens_details": {
"reasoning_tokens": 0
},
"total_tokens": 33
},
"user": null,
"background": false,
"service_tier": "default",
"store": true,
"provider": "openai"
}
```
### Delete Response
Delete a response permanently
```python lines theme={"dark"}
delete_result = client.responses.delete(
response_id=response_id
)
```
Retrieve and delete are only available for providers that store responses server-side. Providers with a stateless Responses API — [OpenRouter](/docs/ai-gateway/openrouter#supported-apis), for example — do not expose them.
## Chat completions served through the Responses API
Everything above covers requests you send to `/responses`. The AI Gateway also uses the Responses API in the other direction: a **Chat Completions** request to OpenAI or Azure OpenAI can be served through the provider's Responses API upstream and converted back before it reaches you.
This exists because OpenAI exposes a model's reasoning only through its Responses API — a plain Chat Completions call to `gpt-5.x` cannot return it. Your client stays on the Chat Completions contract, so your code does not change.
Provided the model supports the Responses API, any one of these turns it on:
| Trigger | Example |
| ------------------------------- | --------------------------------------------------------------------------- |
| The request asks for reasoning | `reasoning_effort` set to anything other than `none`, or a `thinking` field |
| You opt in with a header | `x-tfy-openai-use-responses-api: true` |
| The model only speaks Responses | `codex-mini`, `gpt-5.5-pro` |
That last row is worth calling out: responses-only models used to be rejected on `/chat/completions`. They now work, so you can reach them through the same OpenAI SDK client as every other model.
The model's thinking comes back as `message.thinking_blocks`, alongside a plain-text `message.reasoning_content` summary. Replay the assistant message unchanged on the next turn to continue the reasoning chain.
Backend pinning does **not** apply on this path. Unlike `/responses`, a Chat Completions request that replays `thinking_blocks` to a [virtual model](/docs/ai-gateway/virtual-model) is load-balanced normally, so the replay can land on a different backend than the one that produced the reasoning — and that backend will reject the ciphertext. Point reasoning traffic at a single model, or use `/responses`, if you need the reasoning chain preserved across turns.
Some Chat Completions parameters have no Responses equivalent — including `stop`, `seed`, `logprobs`, and the sampling penalties — and are dropped when a request is served this way. Azure OpenAI also requires an `api-version` of `2025-03-01-preview` or newer.
For the full behaviour — the response shape, multi-turn replay, the complete list of dropped parameters, and the Azure requirements — see [Reasoning on OpenAI and Azure OpenAI](/docs/ai-gateway/chat-completions-advanced#reasoning-on-openai-and-azure-openai).
## Virtual Models
You can call `/responses` on a [virtual model](/docs/ai-gateway/virtual-model) and let the AI Gateway load-balance across several backends. Because a conversation is stored on the backend that created it, the AI Gateway pins every follow-up turn to that same backend automatically — the first turn load-balances, and anything you chain off it with `previous_response_id` goes back to where it started.
The one thing to know as a client: on a virtual model the AI Gateway returns its own response `id` prefixed with `resp_tfyv1-`, which encodes the origin backend. Pass it back verbatim and treat it as opaque; the provider's raw id is not exposed.
Send `store: false` if you'd rather manage history yourself. Note that this does not guarantee load balancing on every turn: if your `input` replays reasoning items carrying `encrypted_content` the AI Gateway minted, that turn is still pinned to the backend that produced it, because reasoning ciphertext is only valid at the model that created it.
For the full behaviour — what pinning changes about routing, how stateless turns behave, and the errors you may see — see [Responses API conversations](/docs/ai-gateway/virtual-model#responses-api-conversations).