> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.

# OTEL Buffer

> Buffer AI Gateway traces on disk in your gateway plane to improve reliability of traces, and optionally send them to other OpenTelemetry collectors.

The OTEL buffer runs next to the AI Gateway in your gateway plane. It stores gateway traces on disk and forwards them to the TrueFoundry control plane, so traces are not lost when the control plane is briefly unreachable. It can also send the same traces to your own OpenTelemetry collectors.

```mermaid theme={"dark"}
flowchart LR
  subgraph GP["Gateway plane"]
    direction LR
    GW["AI Gateway"] -- "traces" --> BUF[("OTEL Buffer")]
  end

  BUF --> COL["TrueFoundry<br/>control plane"]
  BUF -. "configurable" .-> EXT["OTel Collector"]

  classDef box fill:#F4F4F5,stroke:#52525B,color:#18181B
  classDef buffer fill:#E4E4E7,stroke:#18181B,stroke-width:2px,color:#18181B
  class GW,COL,EXT box
  class BUF buffer
  style GP fill:transparent,stroke:#A1A1AA,stroke-dasharray:4 3,color:#71717A
```

## Enable the OTEL buffer

Set `enableOtelBuffer` in your values file:

```yaml truefoundry-values.yaml theme={"dark"}
tfy-llm-gateway:
  enableOtelBuffer: true
```

The gateway now exports traces to the OTEL buffer, and the buffer forwards them to the control plane. Everything else is set up for you.

## Send traces to other collectors

Add destinations under `tfy-otel-buffer.additionalSinks`. They receive the same traces as the control plane.

<Steps>
  <Step title="Store the collector token in a Secret">
    ```bash theme={"dark"}
    kubectl create secret generic tfy-otel-buffer-sink-credentials -n truefoundry \
      --from-literal=COLLECTOR_TOKEN='<YOUR-TOKEN>'
    ```
  </Step>

  <Step title="Add the sink to your values file">
    ```yaml truefoundry-values.yaml theme={"dark"}
    tfy-llm-gateway:
      enableOtelBuffer: true
      tfy-otel-buffer:
        additionalSinks:
          my_collector:
            type: opentelemetry
            protocol:
              type: http
              uri: https://otel-collector.example.com/v1/traces
              method: post
              compression: gzip
              encoding:
                codec: otlp
              request:
                headers:
                  Authorization: "Bearer SECRET[additional_sink_credentials.COLLECTOR_TOKEN]"
    ```

    The traces now appear both in TrueFoundry and in your collector.
  </Step>
</Steps>

## FAQ

<AccordionGroup>
  <Accordion title="Which collectors can I send traces to?">
    Any collector that accepts OTLP over HTTP. Use `protocol.type: http` and the collector's `/v1/traces` endpoint. OTLP over gRPC is not supported in the current TrueFoundry release.
  </Accordion>

  <Accordion title="How do I add or rotate a credential?">
    Add one key per credential to the `tfy-otel-buffer-sink-credentials` Secret and reference it in the sink as `SECRET[additional_sink_credentials.<KEY>]`, for example in a header.

    After you change a key, restart the buffer so it picks up the new value:

    ```bash theme={"dark"}
    kubectl rollout restart statefulset tfy-otel-buffer -n truefoundry
    ```
  </Accordion>

  <Accordion title="What happens when one of my collectors is down?">
    Each destination buffers and retries on its own, so an unreachable collector does not delay traces to the control plane or to your other collectors.
  </Accordion>

  <Accordion title="How much disk can additional sinks use?">
    The disk buffers of all additional sinks together can use up to 5 GiB per replica. This limit can be increased by increasing the buffer volume size.
  </Accordion>
</AccordionGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.