Globally Distributed SAAS Gateway

Learn about Truefoundry’s globally distributed AI Gateway infrastructure deployed across multiple regions and cloud providers

Truefoundry’s globally distributed gateway is deployed across multiple regions and cloud providers to ensure high availability, low latency, and resilience against regional or cloud provider disruptions. The global gateway URL is https://gateway.truefoundry.ai.

gateway.truefoundry.ai is the unified endpoint for both the AI Model Gateway and the MCP Gateway.

Whether you are routing LLM inference requests (OpenAI-compatible API, etc.) or connecting to MCP (Model Context Protocol) servers, all traffic goes through the same globally distributed infrastructure. This means MCP Gateway deployments benefit from the same multi-region, multi-cloud availability described on this page.

Features

1. Globally Distributed: Deployed across more than 12 regions around the globe and across 3 multiple cloud providers for maximum availability while minimizing latency.
2. Automated Failover: All traffic is routed to the nearest gateway for minimum latency. In case of regional downtime, traffic is automatically routed to closest healthy regions ensuring uninterrupted service.
3. Multi-Cloud Deployment: Distributed across multiple cloud providers to be tolerant to cloud provider-specific disruptions.
4. Data Encryption: Data is encrypted at rest and in transit.
5. Compliance: Truefoundry Infrastructure is SOC2, ISO27001, GDPR, and HIPAA compliant

Architecture

The SAAS global deployment follows the same Gateway Plane Architecture used across all Truefoundry deployments. It consists of two key components:

• Control Plane — Manages all gateway configuration including models, users, teams, virtual accounts, rate-limiting, and routing configs. The SAAS control plane is hosted in Ireland (Europe).
• Gateway Planes — Stateless, horizontally scalable gateway instances that handle all production traffic (LLM requests, MCP requests, etc.). These are deployed across the regions listed in the Regional Deployments section below.

The gateway planes subscribe to the control plane for configuration updates via NATS and perform all authentication, authorization, rate-limiting, and load-balancing checks in-memory with no external calls in the request path. For a detailed breakdown of the request flow, performance benchmarks, and FAQs, see the Gateway Plane Architecture page.

The specific regions and locations where gateway planes are deployed are subject to change based on Truefoundry’s internal infrastructure needs. Regions may be added, removed, or relocated without prior notice.

Global Deployment

For most use cases, we recommend using the global endpoint which automatically routes to the nearest healthy gateway:

Deployment	Global Endpoint
Global (Auto-routed)	https://gateway.truefoundry.ai

Using the global URL (https://gateway.truefoundry.ai) is recommended as it automatically routes your requests to the nearest healthy gateway instance based on your geographic location to give you the minimum possible latency.

Regional Deployments

Each of the different gateway regions also have their own URLs as mentioned below. While you can use them for testing, we do not recommend pointing to them in production since they can be changed at any point by us if we are shifting regions or adding new regions.

Please do not use the regional endpoints in production since they are subject to change at any point without notice.

We recommend using the global endpoint (https://gateway.truefoundry.ai) for production use. or the region specific endpoint described in the Multi-regional Deployments section.

Physical Location	Cloud Provider	Regional Endpoint
North Virginia, United States - (ORF)	AWS	https://orf.gateway.truefoundry.ai
Toronto, Canada - (YYZ)	GCP	https://yyz.gateway.truefoundry.ai
San Francisco, United States - (SFO)	Azure	https://sfo.gateway.truefoundry.ai
Dallas, Texas, United States - (DFW)	GCP	https://dfw.gateway.truefoundry.ai
London, United Kingdom - (LHR)	AWS	https://lhr.gateway.truefoundry.ai
Madrid, Spain - (MAD)	GCP	https://mad.gateway.truefoundry.ai
Gavle, Sweden - (GVX)	Azure	https://gvx.gateway.truefoundry.ai
Sao Paulo, Brazil - (GRU)	GCP	https://gru.gateway.truefoundry.ai
Cape Town, South Africa - (CPT)	AWS	https://cpt.gateway.truefoundry.ai
Doha, Qatar - (DIA)	GCP	https://dia.gateway.truefoundry.ai
Mumbai, India - (BOM)	AWS	https://bom.gateway.truefoundry.ai
Singapore, Singapore - (SIN)	AWS	https://sin.gateway.truefoundry.ai
Melbourne, Australia - (MEL)	AWS	https://mel.gateway.truefoundry.ai
Sydney, Australia - (SYD)	AWS	https://syd.gateway.truefoundry.ai

Multi-regional Deployments

Multi-regional endpoints automatically route your requests to the closest healthy gateway within a specific geographic region. If all regional locations are unavailable, traffic is routed to the designated fallback regions.

Region	Multi-regional Endpoint	Primary Locations	Fallback Locations
United States	https://us.gateway.truefoundry.ai	North Virginia (ORF), San Francisco (SFO), Dallas (DFW)	Toronto, Canada (YYZ)
Europe	https://eu.gateway.truefoundry.ai	London (LHR), Madrid (MAD), Gavle (GVX)	Doha, Qatar (DIA)
Australia	https://au.gateway.truefoundry.ai	Sydney (SYD), Melbourne (MEL)	Singapore (SIN)

Gateway Status Monitoring

Check Gateway Status

To track the status of each gateway deployment and receive real-time updates on service availability, visit our status page:

Gateway Status Page: status.truefoundry.com

You can expand the AI Gateway section to see per-region uptime:

Subscribe to Status Updates

Stay informed about gateway availability by subscribing to status notifications:

1. Visit the Gateway Status Page
2. Click the Get Updates button in the top right
3. Choose your preferred notification method:
   • Email notifications
   • RSS Feed
   • On a custom webhook

Connecting Your Models or MCP Servers to the Gateway

When you use the SAAS gateway to proxy requests to models or MCP servers hosted within your own infrastructure (e.g. inside a VPC), the gateway needs a network path to reach those endpoints. Depending on your security requirements and infrastructure setup, there are several approaches to establish this connectivity.

Option 1: IP Allowlisting

The simplest approach is to expose your model or MCP server endpoint publicly and restrict inbound access to only the Truefoundry gateway NAT IPs. Since the gateway is globally distributed, you will need to allowlist the IPs from all regions to ensure requests can reach your endpoint regardless of which gateway instance handles them.

How it works:

1. Expose your model or MCP server on a public IP or domain.
2. Configure your firewall or security group to only allow inbound traffic from the Truefoundry gateway NAT IPs.
3. The gateway authenticates and routes requests to your endpoint over the public internet.

The complete and up-to-date list of gateway NAT IPs is available as a JSON file. Always use this JSON as the source of truth for your firewall configuration, as IPs may be added or changed when new regions are deployed.

JSON endpoint: https://assets.production.truefoundry.com/nat_ip.json

Tradeoffs:

• Simplest to set up — no additional infrastructure required on your end.
• Your endpoint is technically exposed to the public internet, but access is restricted to known gateway IPs.
• You need to keep your firewall rules in sync if gateway IPs change (use the JSON endpoint above to automate this).

Option 2: Reverse Tunnel Agent (Coming Soon)

This feature is coming soon and is not yet available. The information below describes the planned approach.

Instead of opening inbound ports, you can deploy a lightweight tunnel agent inside your VPC that initiates an outbound connection to the Truefoundry relay infrastructure. Since the connection originates from within your network, it punches through NAT and firewalls without requiring any inbound firewall changes.

How it works:

1. Truefoundry provides a small agent binary or container image.
2. You deploy the agent inside your VPC alongside your model or MCP server.
3. The agent establishes a persistent outbound WebSocket or HTTP/2 connection to the Truefoundry relay infrastructure.
4. When the gateway needs to reach your endpoint, it routes traffic through that established tunnel — no inbound ports required.

Points to note:

• Requires deploying and running the tunnel agent in your infrastructure.
• All traffic to your endpoint flows through the Truefoundry relay, adding a small amount of latency.

Option 3: Private Link

For organizations that require fully private network connectivity with no public internet exposure, you can establish a Private Link connection between the Truefoundry gateway and your VPC. This creates a private, direct network path that never traverses the public internet.

Private Link connections are region-specific. You will need to set up a Private Link in each cloud region where you need private connectivity to the gateway. Refer to the Regional Deployments table above to identify the regions and cloud providers available, then contact the Truefoundry team to get the specific service endpoint names for your selected regions.

This is only available as part of our higher tier enterprise plans.

AWS PrivateLink

AWS PrivateLink allows you to privately connect your VPC to the Truefoundry gateway without exposing traffic to the public internet.

Prerequisites:

• An AWS account with a VPC in a supported gateway region
• Appropriate IAM permissions to create VPC endpoints

Steps:

1. Contact the Truefoundry team to get the VPC Endpoint Service Name for your desired region.
2. In the AWS Console, navigate to VPC → Endpoints → Create Endpoint.
3. Select Find service by name and enter the service name provided by Truefoundry.
4. Select the VPC and subnets where your model or MCP server is running.
5. Attach a security group that allows traffic on the required ports.
6. Create the endpoint and wait for the connection to be accepted by Truefoundry.
7. Once active, use the VPC endpoint DNS name as the target for your model or MCP server configuration in the gateway.

# Verify connectivity from within your VPC
curl -X GET "https://<vpc-endpoint-dns>/health"

GCP Private Service Connect

GCP Private Service Connect enables private connectivity from your VPC to the Truefoundry gateway.

Prerequisites:

• A GCP project with a VPC in a supported gateway region
• Appropriate IAM permissions to create Private Service Connect endpoints

Steps:

1. Contact the Truefoundry team to get the Service Attachment URI for your desired region.
2. In the GCP Console, navigate to Network Services → Private Service Connect.
3. Click Create Endpoint and select Published service.
4. Enter the service attachment URI provided by Truefoundry.
5. Select the VPC network and subnetwork where your model or MCP server is running.
6. Assign an internal IP address for the endpoint.
7. Create the endpoint and wait for the connection to be accepted by Truefoundry.
8. Use the assigned internal IP address or configure a DNS entry to route traffic through the Private Service Connect endpoint.

# Verify connectivity from within your VPC
curl -X GET "http://<internal-ip>/health"

Azure Private Link

Azure Private Link provides private connectivity from your VNet to the Truefoundry gateway.

Prerequisites:

• An Azure subscription with a VNet in a supported gateway region
• Appropriate permissions to create private endpoints

Steps:

1. Contact the Truefoundry team to get the Private Link Service Resource ID for your desired region.
2. In the Azure Portal, navigate to Private Link Center → Private Endpoints → Create.
3. Select your subscription, resource group, and region.
4. Under Resource, select Connect to an Azure resource by resource ID or alias and enter the resource ID provided by Truefoundry.
5. Select the VNet and subnet where your model or MCP server is running.
6. Configure DNS integration to automatically create a private DNS zone (recommended).
7. Create the private endpoint and wait for the connection to be approved by Truefoundry.
8. Use the private endpoint’s IP address or DNS name to route traffic.

# Verify connectivity from within your VNet
curl -X GET "http://<private-endpoint-ip>/health"

Tradeoffs:

• Most secure option — traffic never leaves the cloud provider’s private network.
• Requires setup in each region where you need connectivity.
• Involves coordination with the Truefoundry team for service endpoint provisioning.
• May incur additional cloud provider charges for Private Link / Private Service Connect endpoints and data transfer.

Option 4: Self-Host the Gateway

If none of the above options meet your requirements, you can self-host the Truefoundry gateway plane within your own infrastructure. In this setup, the gateway runs inside your VPC and has direct network access to your models and MCP servers — eliminating any need for public exposure or cross-network connectivity. Read more about self-hosting the gateway plane.