Cost Tracking

Set up and manage cost tracking for AI model usage with public and private pricing options.

TrueFoundry AI Gateway makes it easy to track and manage costs for all your AI model usage. This guide walks you through setting up cost tracking, viewing detailed breakdowns, and analyzing your spending.

Setting Up Cost Tracking

To set up cost tracking for your AI models:

1. From the TrueFoundry dashboard, navigate to AI Gateway > Models
2. Select any Model Account
3. Click on the Edit button for an existing model or + Add Model to create a new one
4. In the model configuration screen, you’ll find cost options

Navigating to Model Cost Configuration Interface

TrueFoundry offers two simple ways to track costs:

• Public Cost — Uses provider-published rates from our open-source pricing catalog. Cost per token is auto-populated and updated; ideal for most popular models.
• Private Cost — Lets you set custom pricing for models without public rates, custom contracts, or fine-tuned models.

Public Cost – Automatic Pricing

Setting up public cost

Public cost uses pre-configured pricing based on the provider’s published rates:

• Automatically populated cost per token
• Continuously updated using provider-published rates
• Available for most popular models

How we get public pricing of models?

TrueFoundry maintains an open-source pricing catalog in the truefoundry/models GitHub repository. This repository acts as the pricing database used by AI Gateway for public cost tracking.

The same pricing data is also viewable on the public TrueFoundry Models dashboard, so you can inspect model pricing outside of your workspace as well.

When AI Gateway calculates cost using public pricing, it considers:

• Region-wise pricing: If a model has different rates by deployment region, the matching regional rate is used. For example, AWS Bedrock’s Nova Lite uses different input/output costs per token in us-west-2, us-west-1, eu-central-1, and eu-west-1.
• Tiered pricing: If a provider defines usage tiers (e.g. different rates at different volume thresholds), the applicable tier is selected. For example, Gemini 2.5 Pro has base rates and higher rates from 200K tokens; Gemini 1.5 Flash has tiers starting at 128K tokens.

This helps ensure tracked costs align more closely with how providers bill in real-world scenarios.

Pricing units: tokens, characters, and audio seconds

Not every model is billed per token, so the catalog carries a different rate field depending on what the provider actually charges for:

Model type	Catalog field	Billed on
Chat, completion, embedding	input_cost_per_token, output_cost_per_token	Tokens in and out
Text-to-speech	input_cost_per_character	Characters of input text
Speech-to-text	input_cost_per_second	Seconds of audio

The catalog stores audio rates per single character and per single second. The console normalises them for display, showing per 1M characters and per minute of audio, so a rate that reads 0.0001 in the catalog appears as $100 per 1M characters in the cost UI.

Where the billed quantity comes from differs between the two audio directions, and it matters:

• Text-to-speech counts the characters in your request text, as Unicode code points, before the request is sent upstream.
• Speech-to-text takes the audio duration from the provider’s response. If a provider does not report a duration, no billable seconds are recorded and the request is costed at zero.

Audio cost tracking is wired up for ElevenLabs and Google Vertex text-to-speech and speech-to-text. Other audio providers routed through the gateway record usage but not per-character or per-second cost. Private (custom) audio pricing can currently be entered for Google Vertex only.

Private Cost – Custom Pricing

Setting up private cost

Private cost lets you set custom pricing for:

• Models without public pricing
• Custom pricing contracts
• Fine-tuned models

Viewing Your Costs

Once set up, you can easily view and analyze your costs in the Metrics section. Go to AI Gateway > AI Monitoring > Metrics.

Metrics page overview

The Metrics page shows your total costs, usage trends, and provides interactive filters to analyze your spending patterns by users, virtual accounts, and teams.

Cost Breakdowns

View your costs from different perspectives with a single click:

Cost by user

Cost by model

Cost by team

Each view helps you understand different aspects of your AI usage:

• User view: Identify high-usage individuals
• Model view: See which models cost the most
• Team view: Track department or project spending

Cost Attribution with Metadata

Beyond the built-in user, model, and team breakdowns, you can use custom metadata to build fine-grained cost attribution tailored to your organization’s structure — by application, environment, customer, cost center, or any other dimension.

Automatic Metadata for Cost Attribution

TrueFoundry can automatically inject metadata into requests, enabling cost attribution without any client-side changes:

1. Tag virtual accounts

   Assign tags to your virtual accounts (e.g., application, environment, cost_center). These tags are automatically injected as metadata on every request made with that account’s token, giving you per-application and per-environment cost breakdowns without modifying any code.

2. Associate PATs with teams

   When a PAT is associated with a team, and that team has tags configured, those tags are automatically added to every request. This provides automatic team-level cost attribution for individual users. Admins can mandate team selection to ensure every PAT is tied to a team.

3. Enforce metadata with validation

   Use the Metadata Validation guardrail to require specific metadata keys on every request. For example, mandate that every request includes a cost_center or project_id key — requests missing required metadata are rejected before reaching the model, ensuring complete cost attribution across your organization.

Viewing Costs by Metadata

Once metadata is attached to requests (either manually or via automatic injection), you can filter and group cost data by any metadata key in the Metrics dashboard. For example:

• Group by application to see cost per service
• Group by environment to compare staging vs. production spend
• Group by customer_id to track per-customer costs for chargeback

When exporting cost data, you can include metadata keys in the groupBy fields to get detailed breakdowns in your exported reports.

Exporting Cost Data

Downloading cost data

Need to analyze your data further? Easily export it:

1. Go to the Metrics section
2. Click on the 3 dots button and then click on Download Raw Data
3. Choose the fields you want to groupBy the data

Custom Grouping Options

You can customize how data is grouped in your exports. Simply select the fields you want to group by, such as username, model_name, or teams, to get exactly the data organization you need for your analysis.

Custom grouping options