Routing Metrics

Query Gateway routing, ratelimit, and budget rule application metrics via API.

The Gateway Routing Metrics Query API provides a flexible way to query routing, ratelimit, and budget rule applications: which rule matched a request, where the request was routed, and what happened. You can retrieve either distribution (aggregated) or timeseries results with powerful filtering and grouping.

This page covers datasource: "configMetrics" (the internal literal). The UI sometimes refers to this as Gateway Routing metrics. For other datasources, see the sibling pages for Model, MCP, Guardrail, Cache, and Agent metrics.

All requests go to a single endpoint:

POST https://{your_control_plane_url}/api/svc/v1/llm-gateway/metrics/query

Send JSON with Authorization: Bearer <your_api_key> and Content-Type: application/json.

Access control

Access to metrics is governed by the data access rules configured by your tenant. The server applies these rules automatically based on the caller’s identity—you don’t pass any RBAC or scoping fields in the request. What a caller can query (their own data, their team’s data, or tenant-wide data) depends entirely on the rules an admin has set up.

See Configure Data Access for how these rules are defined and evaluated.

Authentication

Get your API key

Authenticate with your TrueFoundry API key. You can use either a Personal Access Token (PAT) or Virtual Account Token (VAT).

1. Personal Access Token (PAT): Go to Access → Personal Access Tokens in your TrueFoundry dashboard
2. Virtual Account Token (VAT): Go to Access → Virtual Account Tokens (requires admin permissions)

For detailed authentication setup, see our Authentication guide.

Quick start

Distribution query

Loadbalance attempt counts grouped by rule, requested model, target model, and outcome:

import requests

response = requests.post(
    "https://{your_control_plane_url}/api/svc/v1/llm-gateway/metrics/query",
    headers={
        "Authorization": "Bearer <your_api_key>",
        "Content-Type": "application/json"
    },
    json={
        "startTs": "2026-04-21T00:00:00.000Z",
        "endTs": "2026-04-22T00:00:00.000Z",
        "datasource": "configMetrics",
        "type": "distribution",
        "aggregations": [
            {"type": "sum", "column": "loadbalanceTargetAttemptCount"},
            {"type": "avg", "column": "loadbalanceTargetAttemptCount"}
        ],
        "groupBy": ["loadbalanceRuleId", "requestedModel", "targetModel", "status"]
    }
)

print(response.json())

Timeseries query

Hourly routing volume by requested-to-target pair:

import requests

response = requests.post(
    "https://{your_control_plane_url}/api/svc/v1/llm-gateway/metrics/query",
    headers={
        "Authorization": "Bearer <your_api_key>",
        "Content-Type": "application/json"
    },
    json={
        "startTs": "2026-04-21T00:00:00.000Z",
        "endTs": "2026-04-22T00:00:00.000Z",
        "datasource": "configMetrics",
        "type": "timeseries",
        "interval": "1 hour",
        "aggregations": [
            {"type": "sum", "column": "loadbalanceTargetAttemptCount"}
        ],
        "groupBy": ["requestedModel", "targetModel"]
    }
)

print(response.json())

API reference

Post JSON to the endpoint above with Authorization: Bearer <your_api_key> and Content-Type: application/json.

Request parameters

startTs · string · required

ISO 8601 timestamp marking the inclusive lower bound of the query window.

endTs · string · required

ISO 8601 timestamp marking the exclusive upper bound of the query window.

datasource · string · required

The data source to query. Use "configMetrics" for Gateway routing metrics.

type · string · required

The type of query to execute:

• "distribution": returns aggregated rows (one row per groupBy combination).
• "timeseries": returns time-bucketed rows. Requires interval.

aggregations · array

Array of { type, column } objects describing the aggregations to compute. When omitted, only the implicit total = COUNT(*) is returned.

Supported aggregation types

Type	Description
sum	Sum of values
count	Non-null count of the column
countDistinct	Distinct count
min	Minimum value
max	Maximum value
avg	Average
p5, p10, p25, p50, p75, p90, p95, p99, p999	Percentiles (approximate)
rateSum, rateAvg, rateMin, rateMax	Rates normalised by the interval in seconds (timeseries only)
ratePerMinute	Value divided by the interval in minutes (timeseries only)

Supported aggregation columns

Column	Notes
loadbalanceTargetAttemptCount	Number of loadbalance targets attempted. Supports sum, avg, min, max, percentiles

groupBy · array

Array of field names to group results by. Custom metadata keys are supported with a metadata. prefix.

Available group-by fields

Field	Notes
loadbalanceRuleId	The matched loadbalance rule
ratelimitRuleId	The matched ratelimit rule
budgetRuleId	The matched budget rule
requestedModel	The model the caller asked for
targetModel	The model the request was actually routed to
loadbalanceTargetAttemptCount	How many loadbalance targets were attempted
configType	Distinguishes config flavour (e.g. loadbalance, ratelimit, budget)
status	Outcome (e.g. success, fail)
userEmail	Group by user (response key: createdBySubjectSlug)
virtualaccount	Group by virtual account (response key: createdBySubjectSlug)
team	Unnests the Teams array
createdBySubjectType	Distinguishes user vs virtualaccount
metadata.<key>	Group by a custom metadata key

When groupBy contains userEmail (without virtualaccount), the server auto-injects WHERE CreatedBySubjectType = 'user'. virtualaccount alone auto-injects 'virtualaccount'. When both appear, scope it yourself with createdBySubjectType if needed.

filters · array

Array of filter objects, AND-combined. See Filtering below for the full operator reference and the per-field allow-list.

interval · string

Required for timeseries queries. Bucket size as <positive integer> <unit>, where <unit> is one of second, minute, hour, day, week, month, year (with or without a trailing s). Examples: "30 second", "5 minute", "1 hour", "1 day". Compound expressions like "1 hour 30 minute" are rejected.

intervalInSeconds · number · deprecated

Deprecated alias for interval. Accepts a positive integer number of seconds. Prefer interval in new code. If both are provided, interval wins.

Filtering

Filters narrow down the rows that go into each aggregation and group. They are AND-combined; there is no OR-group support. The server enforces a per-field operator allow-list, so the exact subset of operators you can use depends on the field.

Field filters

For standard datasource fields, use fieldName:

{
    "fieldName": "requestedModel",
    "operator": "IN",
    "value": ["gpt-4"]
}

Metadata filters

For custom request-metadata keys, use metadataKey. Works on every datasource:

{
    "metadataKey": "environment",
    "operator": "IN",
    "value": ["production"]
}

Filterable fields and allowed operators

httpStatusCode, errorType, and latencyMs are not filterable on configMetrics. Sending them returns 400 Bad Request with Unsupported gateway config filter name: <field>. Use status in groupBy to see allowed vs blocked or failure outcomes instead.

Field	Type	Allowed operators
loadbalanceRuleId	string	IN, NOT_IN
ratelimitRuleId	string	IN, NOT_IN
budgetRuleId	string	IN, NOT_IN
requestedModel	string	IN, NOT_IN
targetModel	string	IN, NOT_IN
userEmail	string	full string operator set (no IS_NULL)
virtualAccount	string	full string operator set (no IS_NULL)
team	array	ARRAY_HAS_ANY, ARRAY_HAS_NONE
conversationID	string	full string operator set (no IS_NULL)
metadataKey / metadata.<key>	string	full string operator set (no IS_NULL)

Filter operators reference

String field operators

Operator	Description	Example value
EQUAL	Exact match	"[e-mail]"
NOT_EQUAL	Not equal to value	"[e-mail]"
IN	Match any value in the list	["gpt-4", "gpt-3.5-turbo"]
NOT_IN	Exclude values in the list	["deprecated-model"]
STRING_CONTAINS	Contains substring	"prod"
STRING_NOT_CONTAINS	Does not contain substring	"staging"
STRING_STARTS_WITH	Starts with prefix	"prod-"
STRING_NOT_STARTS_WITH	Does not start with prefix	"internal-"
STRING_ENDS_WITH	Ends with suffix	"-v1"
STRING_NOT_ENDS_WITH	Does not end with suffix	"-deprecated"

Array field operators (used by team)

Operator	Description	Example value
ARRAY_HAS_ANY	Match if the array contains any of the values	["team-alpha", "team-beta"]
ARRAY_HAS_NONE	Match if the array contains none of the values	["excluded-team"]

Custom metadata, team unnesting, and combining filters

Custom metadata filtering and grouping. Every datasource supports filtering and grouping by custom request-metadata keys:

• Filter: { "metadataKey": "environment", "operator": "EQUAL", "value": "prod" }
• Group: include "metadata.environment" in the groupBy array.

Metadata fields are treated as strings; use the string field operators above.

Implicit team unnesting. When team is in groupBy (or used as the column of an aggregation), the server transparently UNNESTs the Teams array CTE before applying RBAC. Callers don’t need to do anything extra. Rows whose Teams array is NULL or empty drop out naturally.

Combining multiple filters. Filters are AND-combined:

{
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "distribution",
    "filters": [
        {"fieldName": "requestedModel", "operator": "IN", "value": ["gpt-4"]},
        {"fieldName": "loadbalanceRuleId", "operator": "IN", "value": ["<rule-id>"]},
        {"fieldName": "team", "operator": "ARRAY_HAS_ANY", "value": ["team-alpha"]}
    ],
    "groupBy": ["targetModel", "status"]
}

Query examples

Every example posts a JSON body to the endpoint above. To keep the snippets short, only the json body is shown; the request wrapper is identical to the Quick start.

Distribution examples

Aggregated snapshots of routing rule applications over a time window.

Routing volume by config type and status

Rely on the implicit total count; group by configType and status to see the breakdown of rule applications:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "distribution",
    "groupBy": ["configType", "status"]
}

Attempts distribution per loadbalance rule

Min, average, and p99 of attempts per loadbalance rule, useful when tuning rule fan-out:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "distribution",
    "aggregations": [
        {"type": "min", "column": "loadbalanceTargetAttemptCount"},
        {"type": "avg", "column": "loadbalanceTargetAttemptCount"},
        {"type": "p99", "column": "loadbalanceTargetAttemptCount"}
    ],
    "groupBy": ["loadbalanceRuleId"]
}

Where a specific requested model ends up

Fix requestedModel and see which targetModels the request lands on, with outcome:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "distribution",
    "aggregations": [
        {"type": "sum", "column": "loadbalanceTargetAttemptCount"}
    ],
    "groupBy": ["targetModel", "status"],
    "filters": [
        {"fieldName": "requestedModel", "operator": "IN", "value": ["gpt-4"]}
    ]
}

Activity for a specific ratelimit rule

Filter to a single ratelimitRuleId and group by status to see allowed vs blocked counts for that rule:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "distribution",
    "groupBy": ["status"],
    "filters": [
        {"fieldName": "ratelimitRuleId", "operator": "IN", "value": ["<rule-id>"]}
    ]
}

Failed routings only

Restrict to failed outcomes, broken down by rule:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "distribution",
    "aggregations": [],
    "groupBy": ["loadbalanceRuleId", "configType"],
    "filters": [
        {"fieldName": "loadbalanceRuleId", "operator": "NOT_IN", "value": [""]}
    ]
}

Distinct target models per rule

How many distinct target models a rule is fanning into:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "distribution",
    "aggregations": [
        {"type": "countDistinct", "column": "loadbalanceTargetAttemptCount"}
    ],
    "groupBy": ["loadbalanceRuleId"]
}

Group by team

Routing volume per team:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "distribution",
    "aggregations": [
        {"type": "sum", "column": "loadbalanceTargetAttemptCount"}
    ],
    "groupBy": ["team", "configType"],
    "filters": [
        {"fieldName": "team", "operator": "ARRAY_HAS_ANY", "value": ["team-alpha"]}
    ]
}

Timeseries examples

Time-bucketed routing metrics over a window. Every timeseries query must include interval (or the deprecated intervalInSeconds).

Hourly routing volume by config type

Hourly applications per configType:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "groupBy": ["configType"]
}

Hourly average attempts per rule

Track loadbalance rule pressure over time:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "aggregations": [
        {"type": "avg", "column": "loadbalanceTargetAttemptCount"}
    ],
    "groupBy": ["loadbalanceRuleId"]
}

Hourly routing volume per requested-to-target pair

Useful to confirm a routing change took effect:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "aggregations": [
        {"type": "sum", "column": "loadbalanceTargetAttemptCount"}
    ],
    "groupBy": ["requestedModel", "targetModel"]
}

Hourly outcome breakdown

Volume per status over time:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "groupBy": ["status"]
}

5-minute traffic during an incident

Fine-grained breakdown to investigate a routing change:

json={
    "startTs": "2026-04-21T14:00:00.000Z",
    "endTs": "2026-04-21T16:00:00.000Z",
    "datasource": "configMetrics",
    "type": "timeseries",
    "interval": "5 minute",
    "aggregations": [
        {"type": "sum", "column": "loadbalanceTargetAttemptCount"}
    ],
    "groupBy": ["targetModel"],
    "filters": [
        {"fieldName": "requestedModel", "operator": "IN", "value": ["gpt-4"]}
    ]
}

Daily routings over a week

Daily routing volume across a 7-day window:

json={
    "startTs": "2026-04-14T00:00:00.000Z",
    "endTs": "2026-04-21T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "timeseries",
    "interval": "1 day",
    "groupBy": ["configType"]
}

Hourly activity for a specific ratelimit rule

Watch a single rule’s volume over time:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "configMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "groupBy": ["status"],
    "filters": [
        {"fieldName": "ratelimitRuleId", "operator": "IN", "value": ["<rule-id>"]}
    ]
}

Response format

Every successful response has the same outer shape:

{
  "data": {
    "dataPoints": [
      {
        "startTimestamp": "2026-04-29T12:00:00.000Z",
        "endTimestamp": "2026-04-29T13:00:00.000Z",
        "total": 1234,
        "<aggregationKey>": 0,
        "<groupByKey>": "value-or-null"
      }
    ]
  }
}

• total: implicit COUNT(*) for the row. Always present.
• <aggregationKey>: one key per requested aggregation. The key is <type><Column> in camelCase (e.g. sumLoadbalanceTargetAttemptCount, avgLoadbalanceTargetAttemptCount).
• <groupByKey>: one key per groupBy entry. The key is the lowerCamelCase form of the underlying column. Two special mappings:
  • userEmail and virtualaccount both map to createdBySubjectSlug in the response.
  • team maps to team (the value is a single unnested scalar, not an array). All other groupBy keys preserve their lowerCamelCase name.
• startTimestamp: present only for timeseries responses. Bucket start as an ISO 8601 timestamp string (e.g. "2026-04-29T12:00:00.000Z"). Distribution responses omit it.
• endTimestamp: present only for timeseries responses. Bucket end as an ISO 8601 timestamp string, equal to the next bucket’s startTimestamp (e.g. "2026-04-29T13:00:00.000Z"). Distribution responses omit it.

Distribution response example

{
  "data": {
    "dataPoints": [
      {
        "loadbalanceRuleId": "lb-rule-prod",
        "requestedModel": "gpt-4",
        "targetModel": "gpt-4o",
        "status": "success",
        "total": 1240,
        "sumLoadbalanceTargetAttemptCount": 1260,
        "avgLoadbalanceTargetAttemptCount": 1.016
      },
      {
        "loadbalanceRuleId": "lb-rule-prod",
        "requestedModel": "gpt-4",
        "targetModel": "gpt-4o-mini",
        "status": "success",
        "total": 360,
        "sumLoadbalanceTargetAttemptCount": 412,
        "avgLoadbalanceTargetAttemptCount": 1.144
      }
    ]
  }
}

Timeseries response example

{
  "data": {
    "dataPoints": [
      {
        "startTimestamp": "2026-04-21T00:00:00.000Z",
        "endTimestamp": "2026-04-21T01:00:00.000Z",
        "requestedModel": "gpt-4",
        "targetModel": "gpt-4o",
        "total": 52,
        "sumLoadbalanceTargetAttemptCount": 54
      },
      {
        "startTimestamp": "2026-04-21T01:00:00.000Z",
        "endTimestamp": "2026-04-21T02:00:00.000Z",
        "requestedModel": "gpt-4",
        "targetModel": "gpt-4o",
        "total": 61,
        "sumLoadbalanceTargetAttemptCount": 63
      }
    ]
  }
}

If groupBy is empty or omitted, the response collapses to a single row (or one row per timeseries bucket) summarising every rule application inside the window.

Error responses

A malformed query returns 400 Bad Request:

{
  "statusCode": 400,
  "message": "Invalid query",
  "details": ["..."]
}

Common causes and other status codes

Common causes of 400:

• Unsupported field name for configMetrics. The error message is typically Unsupported gateway config filter name: <field>. Notably, httpStatusCode, errorType, and latencyMs are not filterable on this datasource; use status in groupBy for outcome breakdowns.
• Operator not allowed on this field. For example, STRING_CONTAINS on loadbalanceRuleId (it supports only IN/NOT_IN).
• Missing required value (or wrong shape, e.g. scalar where array is expected for IN).
• Invalid interval format (compound expressions, unrecognised unit, non-positive integer).
• Missing required interval for a timeseries query.

Other status codes:

• 401 Unauthorized: missing or invalid bearer token.
• 403 Forbidden: caller does not have permission for the requested scope.
• 500 Internal Server Error: unexpected server error while executing the query.