Skip to main content

Documentation Index

Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt

Use this file to discover all available pages before exploring further.

The architecture of a TrueFoundry compute plane is as follows:
PolicyDescription
Access required for Azure container registry, storage accountAn azure container registry is used to store the docker images for the platform. A storage account is used to store the model artifacts.
Azure AD application with Reader and Monitoring Reader on AKSReader and monitoring reader permission on AKS is used to access the cluster autoscaler logs in Log Analytics and read azure node pools. User should have access to create Azure AD application.

​
Requirements:

The common requirements to setup compute plane in each of the scenarios is as follows:
  • Billing must be enabled for the Azure subscription.
  • Ensure that Microsoft.Storage resource provider is registered. Check this link for more details.
  • Egress access to container registries - public.ecr.aws, quay.io, ghcr.io, tfy.jfrog.io, docker.io/natsio, nvcr.io, registry.k8s.io so that we can download the docker images for argocd, nats, gpu operator, argo rollouts, argo workflows, istio, keda, etc.
  • We need a domain to map to the service endpoints and certificate to encrypt the traffic. A wildcard domain like *.services.example.com is preferred. TrueFoundry can do path based routing like services.example.com/tfy/*, however, many frontend applications do not support this.
  • Enough quotas for CPU/GPU instances must be present depending on your usecase. You can check and increase quotas at Azure compute quotas
  • Ensure that host encryption is enabled.
  1. The new VPC subnet should have a CIDR range of /24 or larger. Secondary ranges for pods (min /20) and services (min /24) are required. Secondary range can be from a non-routable range. This is to ensure capacity for ~250 instances and 4096 pods.
  2. User/serviceaccount to provision the infrastructure.

​
Setting up compute plane

TrueFoundry compute plane infrastructure is provisioned using OpenTofu/Terraform. You can download the OpenTofu/Terraform code for your exact account by filling up your account details and downloading a script that can be executed on your local machine.
1

Enable Deployment Feature in the Platform (Optional)

To enable the deployment feature which allows you to deploy services through the platform, you need to enable it;
  • In the left hand navigation, go to Settings then Platform Feature Visibility under Preferences
  • Click on Edit button. Then enable the toggle for Enable Deployment
  • Click on Save button.
This will enable the deployment feature in the platform and allow you to create either a control plane and compute plane.
2

Choose to create a new cluster or attach an existing cluster

Go to the platform section in the left panel and click on Clusters. You can click on Create New Cluster or Attach Existing Cluster depending on your use case. Read the requirements and if everything is satisfied, click on Continue.
3

Fill up the form to generate the OpenTofu/Terraform code

A form will be presented with the details for the new cluster to be created. Fill in with your cluster details. Click Submit when done
The key fields to fill up here are:
  • Region - The region and availability zones where you want to create the cluster.
  • Resource Group - The resource group where you want to create the cluster. Chose between New Resource Group or Existing Resource Group depending on your use case.
  • Cluster Name - A name for your cluster.
  • Kubernetes Version - The Kubernetes version for the cluster (e.g. 1.34).
  • Node Pools - Configure CPU and GPU node pools for the cluster. The form comes with sensible defaults (see below) which you can adjust based on your workload requirements. Azure Node Pools configuration form The default node pool configuration is:
    PoolTypeInstance TypeCapacityMinMax
    initial (system)On-DemandStandard_D4ds_v5CPU22
    cpuOn-DemandStandard_D4ds_v5CPU02
    cpu2xOn-DemandStandard_D8ds_v5CPU02
    a10On-DemandStandard_NV6ads_A10_v5GPU02
    t4On-DemandStandard_NC4as_T4_v3GPU02
    The initial pool is the system node pool that runs TrueFoundry platform components (ArgoCD, Istio, tfy-agent, etc.) and must always be on-demand with at least 2 nodes. You can add, remove, or resize the other CPU/GPU pools to match your workload needs. GPU pools can be removed entirely if you don’t plan to run GPU workloads. Make sure you have sufficient Azure compute quotas for the instance types you select.
  • Network Configuration - Choose between New Vnet or Existing Vnet depending on your use case.
  • DNS Configuration - Configure the DNS zone and domains that will point to the cluster’s load balancer. This also provisions a TLS certificate for those domains. Select New DNS Zone or Existing DNS Zone if you want TrueFoundry to manage DNS in Azure. If you use an external DNS provider (e.g., Route53, Cloudflare), you can skip this section.
  • Storage account (container) for OpenTofu/Terraform State - OpenTofu/Terraform state will be stored in this container. It can be a preexisting storage account or a new storage account name. The new storage account will automatically be created by our script.
  • Platform Features - This is to decide which features like BlobStorage, ClusterIntegration using Azure AD and Container Registry will be enabled for your cluster. To read more on how these integrations are used in the platform, please refer to the platform features page.
4

Copy the curl command and execute it on your local machine

You will be presented with a curl command to download and execute the script. The script will take care of installing the pre-requisites, downloading OpenTofu/Terraform code and running it on your local machine to create the cluster. This will take around 40-50 minutes to complete.
5

Verify the cluster is showing as connected in the platform

Once the script is executed, the cluster will be shown as connected in the platform.
6

Create DNS Record

We can get the load-balancer’s IP address by going to the platform section in the bottom left panel under the Clusters section. Under the preferred cluster, you’ll see the load balancer IP address under the Base Domain URL section.Create a DNS record in your Azure DNS Zone or your DNS provider with the following details
Record TypeRecord NameRecord value
A*.tfy.example.comLOADBALANCER_IP_ADDRESS
7

Start deploying workloads to your cluster

You can start by going here

​
Permissions required to create the infrastructure

The IAM user should have the following permissions -
  • Contributor Role to the above Subscription
  • Role Based Access Administrator to the above subscription
  • Either Azure AD Administrator or Azure AD Application Developer role to:
    • Create app registrations and service principals
    • Assign Reader role to AD application for read-only AKS cluster access
    • Assign Monitoring Reader role to applications for cluster monitoring (Ref: How to add Azure admin permission

​
FAQ

If you have your own certificate files (for example, from another certificate provider or self-signed), you can use them directly with TrueFoundry.
  1. Create a Kubernetes secret with your certificate and key, or create a self-signed certificate: 
    # Generate a self-signed certificate
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout tls.key -out tls.crt \
  -subj "/CN=*.example.com" \
  -addext "subjectAltName = DNS:example.com,DNS:*.example.com"

    # Create secret from local certificate files
kubectl create secret tls example-com-tls \
  --cert=path/to/cert/file \
  --key=path/to/key/file \
  -n istio-system
  2. Once the secret is created, head over to the cluster page and navigate to the tfy-istio-ingress add-on. Add the secret name in the tfyGateway.spec.servers[1].tls.credentialName section and ensure that tfyGateway.spec.servers[1].port.protocol is set to HTTPS. Here we are using example-com-tls as the secret name, which contains the certificate and key. 
        servers:
      - <REDACTED>
      - hosts:
          - "*"
        port:
          name: https-tfy-wildcard
          number: 443
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: example-com-tls
Self-signed certificates will cause browser warnings. They should only be used for testing or internal systems. To connect to services with self-signed certificates, you have to pass the CA certificate to verify the SSL certificate.
If you need to add or modify node pools after the cluster is created, you can do so using the Azure CLI. Set the following variables before running the commands:
export RESOURCE_GROUP="<your-resource-group>"
export CLUSTER_NAME="<your-cluster-name>"
export INSTANCE_SIZE="<instance-type>"   # e.g. Standard_D4ds_v5
You can browse available instance types and pricing at azureprice.net.
az aks nodepool add \
    --resource-group $RESOURCE_GROUP \
    --cluster-name $CLUSTER_NAME \
    --name <NODEPOOL_NAME> \
    --enable-cluster-autoscaler \
    --enable-encryption-at-host \
    --node-vm-size $INSTANCE_SIZE \
    --min-count 0 \
    --node-count 0 \
    --node-osdisk-size 100 \
    --max-count 10 \
    --no-wait
New node pools are automatically synced in TrueFoundry if the Azure AD application has Reader access on the AKS cluster.
On-DemandSpot
AvailabilityGuaranteed — no interruptionsCan be reclaimed by Azure at any time
CostStandard pricingUp to 60-90% cheaper
Best forProduction services, databases, platform componentsDev/test workloads, batch jobs, interruptible tasks
The system node pool (initial) must always be on-demand. For user workloads, spot pools can bring significant cost savings if the application can tolerate interruptions. Make sure you have sufficient spot quotas in your region.