Agent Metrics

Query Gateway agent invocation metrics for framework, outcome, and latency analytics via API.

The Gateway Agent Metrics Query API provides a flexible way to query agent-framework invocations and their outcomes: which agent ran, on which framework, on which server type, and how long it took. You can retrieve either distribution (aggregated) or timeseries results with powerful filtering and grouping.

This page covers datasource: "agentMetrics". For other datasources, see the sibling pages for Model, MCP, Guardrail, Cache, and Routing metrics.

All requests go to a single endpoint:

POST https://{your_control_plane_url}/api/svc/v1/llm-gateway/metrics/query

Send JSON with Authorization: Bearer <your_api_key> and Content-Type: application/json.

Access control

Access to metrics is governed by the data access rules configured by your tenant. The server applies these rules automatically based on the caller’s identity—you don’t pass any RBAC or scoping fields in the request. What a caller can query (their own data, their team’s data, or tenant-wide data) depends entirely on the rules an admin has set up.

See Configure Data Access for how these rules are defined and evaluated.

Authentication

Get your API key

Authenticate with your TrueFoundry API key. You can use either a Personal Access Token (PAT) or Virtual Account Token (VAT).

1. Personal Access Token (PAT): Go to Access → Personal Access Tokens in your TrueFoundry dashboard
2. Virtual Account Token (VAT): Go to Access → Virtual Account Tokens (requires admin permissions)

For detailed authentication setup, see our Authentication guide.

Quick start

By default, agent metrics include every Gateway request, not just agent invocations. Rows that didn’t go through an agent will have agentName, agentFramework, and agentServerType set to null and surface in groupBy output as null buckets. The IS_NULL operator is not supported on these three fields; to scope a query to a known set of agents, filter with {"fieldName": "agentName", "operator": "IN", "value": ["<your-agent>"]} or use STRING_* operators.

Distribution query

p50 and p99 latency per agent:

import requests

response = requests.post(
    "https://{your_control_plane_url}/api/svc/v1/llm-gateway/metrics/query",
    headers={
        "Authorization": "Bearer <your_api_key>",
        "Content-Type": "application/json"
    },
    json={
        "startTs": "2026-04-21T00:00:00.000Z",
        "endTs": "2026-04-22T00:00:00.000Z",
        "datasource": "agentMetrics",
        "type": "distribution",
        "aggregations": [
            {"type": "p50", "column": "latencyMs"},
            {"type": "p99", "column": "latencyMs"}
        ],
        "groupBy": ["agentName"]
    }
)

print(response.json())

Timeseries query

Hourly p99 latency per agent, restricted to failed invocations:

import requests

response = requests.post(
    "https://{your_control_plane_url}/api/svc/v1/llm-gateway/metrics/query",
    headers={
        "Authorization": "Bearer <your_api_key>",
        "Content-Type": "application/json"
    },
    json={
        "startTs": "2026-04-21T00:00:00.000Z",
        "endTs": "2026-04-22T00:00:00.000Z",
        "datasource": "agentMetrics",
        "type": "timeseries",
        "interval": "1 hour",
        "aggregations": [
            {"type": "p99", "column": "latencyMs"}
        ],
        "groupBy": ["agentName"],
        "filters": [
            {"fieldName": "isFailure", "operator": "EQUAL", "value": true}
        ]
    }
)

print(response.json())

API reference

Post JSON to the endpoint above with Authorization: Bearer <your_api_key> and Content-Type: application/json.

Request parameters

startTs · string · required

ISO 8601 timestamp marking the inclusive lower bound of the query window.

endTs · string · required

ISO 8601 timestamp marking the exclusive upper bound of the query window.

datasource · string · required

The data source to query. Use "agentMetrics" for Gateway agent metrics.

type · string · required

The type of query to execute:

• "distribution": returns aggregated rows (one row per groupBy combination).
• "timeseries": returns time-bucketed rows. Requires interval.

aggregations · array

Array of { type, column } objects describing the aggregations to compute. When omitted, only the implicit total = COUNT(*) is returned.

Supported aggregation types

Type	Description
sum	Sum of values
count	Non-null count of the column
countDistinct	Distinct count
min	Minimum value
max	Maximum value
avg	Average
p5, p10, p25, p50, p75, p90, p95, p99, p999	Percentiles (approximate)
rateSum, rateAvg, rateMin, rateMax	Rates normalised by the interval in seconds (timeseries only)
ratePerMinute	Value divided by the interval in minutes (timeseries only)

Supported aggregation columns

Column	Supported aggregation types	Notes
latencyMs	all scalar and all percentiles	Agent invocation latency in ms

groupBy · array

Array of field names to group results by. Custom metadata keys are supported with a metadata. prefix.

Available group-by fields

Field	Notes
agentName	Agent identifier
agentFramework	Framework, e.g. langgraph, crewai
agentServerType	Transport, e.g. sse, streamable-http
isFailure	Boolean: true for failed invocations. GroupBy is permitted but rarely useful (only two values); filtering is the primary use.
httpStatusCode	Raw HTTP status code (NULL for rows without one). Frontends typically render NULL as “unknown” and numerics as HTTP: <code>.
userEmail	Group by user (response key: createdBySubjectSlug)
virtualaccount	Group by virtual account (response key: createdBySubjectSlug)
team	Unnests the Teams array
createdBySubjectType	Distinguishes user vs virtualaccount
metadata.<key>	Group by a custom metadata key

When groupBy contains userEmail (without virtualaccount), the server auto-injects WHERE CreatedBySubjectType = 'user'. virtualaccount alone auto-injects 'virtualaccount'. When both appear, scope it yourself with createdBySubjectType if needed.

filters · array

Array of filter objects, AND-combined. See Filtering below for the full operator reference and the per-field allow-list.

interval · string

Required for timeseries queries. Bucket size as <positive integer> <unit>, where <unit> is one of second, minute, hour, day, week, month, year (with or without a trailing s). Examples: "30 second", "5 minute", "1 hour", "1 day". Compound expressions like "1 hour 30 minute" are rejected.

intervalInSeconds · number · deprecated

Deprecated alias for interval. Accepts a positive integer number of seconds. Prefer interval in new code. If both are provided, interval wins.

Filtering

Filters narrow down the rows that go into each aggregation and group. They are AND-combined; there is no OR-group support. The server enforces a per-field operator allow-list, so the exact subset of operators you can use depends on the field.

Field filters

For standard datasource fields, use fieldName:

{
    "fieldName": "agentName",
    "operator": "IN",
    "value": ["support-bot", "research-agent"]
}

Metadata filters

For custom request-metadata keys, use metadataKey. Works on every datasource:

{
    "metadataKey": "environment",
    "operator": "IN",
    "value": ["production"]
}

Filterable fields and allowed operators

The agent-specific string fields (agentName, agentFramework, agentServerType) accept the narrow set IN, NOT_IN, STRING_CONTAINS, STRING_STARTS_WITH, STRING_ENDS_WITH. IS_NULL is not supported on these fields; sending it returns 400 Bad Request with Field "agentName" does not support operator "IS_NULL". EQUAL and NOT_EQUAL are only supported on subject fields (userEmail, virtualAccount) and conversationID.

Field	Type	Allowed operators
agentName	string	IN, NOT_IN, STRING_CONTAINS, STRING_STARTS_WITH, STRING_ENDS_WITH
agentFramework	string	IN, NOT_IN, STRING_CONTAINS, STRING_STARTS_WITH, STRING_ENDS_WITH
agentServerType	string	IN, NOT_IN, STRING_CONTAINS, STRING_STARTS_WITH, STRING_ENDS_WITH
isFailure	boolean	EQUAL
httpStatusCode	number	EQUAL, NOT_EQUAL, IN, NOT_IN, GREATER_THAN, LESS_THAN, GREATER_THAN_EQUAL, LESS_THAN_EQUAL, IS_NULL
userEmail	string	full string operator set (no IS_NULL)
virtualAccount	string	full string operator set (no IS_NULL)
team	array	ARRAY_HAS_ANY, ARRAY_HAS_NONE
latencyMs	number	GREATER_THAN, LESS_THAN, GREATER_THAN_EQUAL, LESS_THAN_EQUAL, BETWEEN
conversationID	string	full string operator set
metadataKey / metadata.<key>	string	full string operator set

Filter operators reference

String field operators

Operator	Description	Example value
EQUAL	Exact match	"[e-mail]"
NOT_EQUAL	Not equal to value	"[e-mail]"
IN	Match any value in the list	["langgraph", "crewai"]
NOT_IN	Exclude values in the list	["deprecated-framework"]
STRING_CONTAINS	Contains substring	"graph"
STRING_NOT_CONTAINS	Does not contain substring	"internal"
STRING_STARTS_WITH	Starts with prefix	"prod-"
STRING_NOT_STARTS_WITH	Does not start with prefix	"internal-"
STRING_ENDS_WITH	Ends with suffix	"-v1"
STRING_NOT_ENDS_WITH	Does not end with suffix	"-deprecated"
IS_NULL	true matches rows where the field is unset; false matches rows where it is set	true

Numeric field operators

Operator	Description	Example value
EQUAL	Exact match	200
NOT_EQUAL	Not equal to value	200
IN	Match any value in the list	[200, 404, 500]
NOT_IN	Exclude values in the list	[200]
GREATER_THAN	Strictly greater than	499
LESS_THAN	Strictly less than	300
GREATER_THAN_EQUAL	Greater than or equal to	400
LESS_THAN_EQUAL	Less than or equal to	299
BETWEEN	Between two values (inclusive)	[500, 5000]
IS_NULL	true matches rows where the field is unset; false matches rows where it is set	true

Boolean field operators

Operator	Description	Example value
EQUAL	Exact match	true

Array field operators (used by team)

Operator	Description	Example value
ARRAY_HAS_ANY	Match if the array contains any of the values	["team-alpha", "team-beta"]
ARRAY_HAS_NONE	Match if the array contains none of the values	["excluded-team"]

Custom metadata, team unnesting, and combining filters

Custom metadata filtering and grouping. Every datasource supports filtering and grouping by custom request-metadata keys:

• Filter: { "metadataKey": "environment", "operator": "EQUAL", "value": "prod" }
• Group: include "metadata.environment" in the groupBy array.

Metadata fields are treated as strings; use the string field operators above.

Implicit team unnesting. When team is in groupBy (or used as the column of an aggregation), the server transparently UNNESTs the Teams array CTE before applying RBAC. Callers don’t need to do anything extra. Rows whose Teams array is NULL or empty drop out naturally.

Combining multiple filters. Filters are AND-combined:

{
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "distribution",
    "filters": [
        {"fieldName": "agentFramework", "operator": "IN", "value": ["langgraph"]},
        {"fieldName": "agentName", "operator": "IN", "value": ["support-bot", "research-agent"]},
        {"fieldName": "latencyMs", "operator": "BETWEEN", "value": [100, 10000]},
        {"fieldName": "team", "operator": "ARRAY_HAS_ANY", "value": ["team-alpha"]}
    ],
    "groupBy": ["agentName", "isFailure"]
}

Query examples

Every example posts a JSON body to the endpoint above. To keep the snippets short, only the json body is shown; the request wrapper is identical to the Quick start.

Agent metrics include every Gateway request by default. Rows that didn’t go through an agent will have agentName, agentFramework, and agentServerType set to null and show up as null buckets in groupBy output. IS_NULL is not supported on these three fields; to scope to specific known agents, use agentName IN [...] or one of the STRING_* operators.

Distribution examples

Volume by framework

Counts grouped by agent framework, useful for capacity planning across langgraph, crewai, etc.:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "distribution",
    "groupBy": ["agentFramework"]
}

Failure rate per agent

Counts grouped by agent and outcome. Surfaces flaky agents:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "distribution",
    "groupBy": ["agentName", "isFailure"]
}

Latency percentiles by framework

p50, p90, and p99 latency grouped by framework:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "distribution",
    "aggregations": [
        {"type": "p50", "column": "latencyMs"},
        {"type": "p90", "column": "latencyMs"},
        {"type": "p99", "column": "latencyMs"}
    ],
    "groupBy": ["agentFramework"]
}

HTTP status code distribution

Group by raw httpStatusCode; rows without one show up as null (rendered as “unknown” in dashboards):

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "distribution",
    "groupBy": ["httpStatusCode"]
}

Server-type and framework mix

Cross-tab of transport vs framework:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "distribution",
    "groupBy": ["agentServerType", "agentFramework"]
}

Failures only by agent

Restrict to failed invocations:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "distribution",
    "aggregations": [
        {"type": "p99", "column": "latencyMs"}
    ],
    "groupBy": ["agentName"],
    "filters": [
        {"fieldName": "isFailure", "operator": "EQUAL", "value": true}
    ]
}

Slow invocations

Volume of invocations slower than 30 seconds, grouped by agent:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "distribution",
    "aggregations": [
        {"type": "count", "column": "agentName"}
    ],
    "groupBy": ["agentName"],
    "filters": [
        {"fieldName": "latencyMs", "operator": "GREATER_THAN", "value": 30000}
    ]
}

Scope to a known set of agents

Use agentName IN [...] to restrict the query to specific agents you care about:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "distribution",
    "aggregations": [
        {"type": "p99", "column": "latencyMs"}
    ],
    "groupBy": ["agentName"],
    "filters": [
        {"fieldName": "agentName", "operator": "IN", "value": ["support-bot", "research-agent"]}
    ]
}

Timeseries examples

Every timeseries query must include interval (or the deprecated intervalInSeconds). Buckets are expressed as <positive integer> <unit> strings like "5 minute", "1 hour", or "1 day".

Hourly volume per agent

Per-agent invocation count over time:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "groupBy": ["agentName"]
}

Hourly p99 latency per agent

Watch latency regressions per agent:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "aggregations": [
        {"type": "p99", "column": "latencyMs"}
    ],
    "groupBy": ["agentName"]
}

Hourly failures over time

Track failures per agent bucket-by-bucket:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "aggregations": [
        {"type": "count", "column": "agentName"}
    ],
    "groupBy": ["agentName"],
    "filters": [
        {"fieldName": "isFailure", "operator": "EQUAL", "value": true}
    ]
}

Hourly framework mix

Per-framework volume over time:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "groupBy": ["agentFramework"]
}

5-minute latency in an incident window

Fine-grained breakdown to investigate a regression:

json={
    "startTs": "2026-04-21T14:00:00.000Z",
    "endTs": "2026-04-21T16:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "timeseries",
    "interval": "5 minute",
    "aggregations": [
        {"type": "p99", "column": "latencyMs"}
    ],
    "groupBy": ["agentName"]
}

Daily invocations over a week

Daily agent traffic across a 7-day window:

json={
    "startTs": "2026-04-14T00:00:00.000Z",
    "endTs": "2026-04-21T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "timeseries",
    "interval": "1 day",
    "groupBy": ["agentName"]
}

Hourly per-team activity

Per-team agent usage over time:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "groupBy": ["team"],
    "filters": [
        {"fieldName": "team", "operator": "ARRAY_HAS_ANY", "value": ["team-alpha", "team-beta"]}
    ]
}

Hourly p99 latency for a known set of agents

Use agentName IN [...] to scope the timeseries to specific agents:

json={
    "startTs": "2026-04-21T00:00:00.000Z",
    "endTs": "2026-04-22T00:00:00.000Z",
    "datasource": "agentMetrics",
    "type": "timeseries",
    "interval": "1 hour",
    "aggregations": [
        {"type": "p99", "column": "latencyMs"}
    ],
    "groupBy": ["agentName"],
    "filters": [
        {"fieldName": "agentName", "operator": "IN", "value": ["support-bot", "research-agent"]}
    ]
}

Response format

Every successful response has the same outer shape:

{
  "data": {
    "dataPoints": [
      {
        "startTimestamp": "2026-04-29T12:00:00.000Z",
        "endTimestamp": "2026-04-29T13:00:00.000Z",
        "total": 1234,
        "<aggregationKey>": 0,
        "<groupByKey>": "value-or-null"
      }
    ]
  }
}

• total: implicit COUNT(*) for the row. Always present.
• <aggregationKey>: one key per requested aggregation. The key is <type><Column> in camelCase (e.g. countAgentName, p99LatencyMs).
• <groupByKey>: one key per groupBy entry. The key is the lowerCamelCase form of the underlying column. Two special mappings:
  • userEmail and virtualaccount both map to createdBySubjectSlug in the response.
  • team maps to team (the value is a single unnested scalar, not an array). All other groupBy keys preserve their lowerCamelCase name.
• startTimestamp / endTimestamp: present only for timeseries responses. Bucket start and end as ISO 8601 timestamp strings; endTimestamp equals the next bucket’s startTimestamp. Distribution responses omit both.

Distribution response example

{
  "data": {
    "dataPoints": [
      {
        "agentName": "support-bot",
        "total": 1240,
        "p50LatencyMs": 1820.5,
        "p99LatencyMs": 8240.0
      },
      {
        "agentName": "research-agent",
        "total": 380,
        "p50LatencyMs": 4220.0,
        "p99LatencyMs": 12480.0
      }
    ]
  }
}

Timeseries response example

{
  "data": {
    "dataPoints": [
      {
        "startTimestamp": "2026-04-21T00:00:00.000Z",
        "endTimestamp": "2026-04-21T01:00:00.000Z",
        "agentName": "support-bot",
        "total": 52,
        "p99LatencyMs": 8100.0
      },
      {
        "startTimestamp": "2026-04-21T01:00:00.000Z",
        "endTimestamp": "2026-04-21T02:00:00.000Z",
        "agentName": "support-bot",
        "total": 61,
        "p99LatencyMs": 8240.0
      }
    ]
  }
}

If groupBy is empty or omitted, the response collapses to a single row (or one row per timeseries bucket) summarising every agent invocation inside the window.

By default the response includes non-agent traffic (rows where agentName is null), since IS_NULL is not a valid operator on agentName. Most dashboards either render these as “unknown agent” or filter them out client-side. To scope server-side to specific known agents, use {"fieldName": "agentName", "operator": "IN", "value": [...]}.

Error responses

A malformed query returns 400 Bad Request:

{
  "statusCode": 400,
  "message": "Invalid query",
  "details": ["..."]
}

Common causes and other status codes

Common causes of 400:

• Operator not allowed on this field, for example, EQUAL or IS_NULL on agentName (it supports only IN, NOT_IN, and the STRING_* operators).
• Missing required value (or wrong shape, e.g. scalar where array is expected for IN / BETWEEN).
• Unknown field name for the datasource.
• Invalid interval format (compound expressions, unrecognised unit, non-positive integer).
• Missing required interval for a timeseries query.

Other status codes:

• 401 Unauthorized: missing or invalid bearer token.
• 403 Forbidden: caller does not have permission for the requested scope.
• 500 Internal Server Error: unexpected server error while executing the query.