How Platform Engineering Leverages Kubernetes and Terraform for Scalable Cloud Infrastructure
Platform engineering uses Terraform to provision the cloud foundation (networks, identity, managed databases, and the Kubernetes clusters themselves) and Kubernetes as the standardized runtime and control plane where applications are deployed, scaled, and governed. A platform team packages both into an internal developer platform: versioned Terraform modules, GitOps-managed cluster configuration, policy as code, and "golden path" templates. Product teams then ship services without filing tickets or re-solving infrastructure problems. Terraform handles infrastructure that changes slowly. Kubernetes handles workloads that change constantly. A well-defined boundary between the two is what makes the system scale.
The rest of this post explains how we approach that split in practice, step by step, with the code patterns we reach for most often.
The Problem Platform Engineering Solves
Most organizations don't set out to build a platform. They arrive at one after a familiar sequence of events:
- Every team writes its own Terraform, so there are five slightly different VPC configurations and no one is sure which is correct.
- Kubernetes clusters drift because someone ran
kubectl applyfrom a laptop six months ago. - Developers wait days for a database or a namespace because only two people understand the provisioning process.
- Security reviews happen at the end of delivery, when fixing problems is most expensive.
The cost here is cognitive load. Team Topologies (Skelton and Pais) describes a platform team's job as reducing the cognitive load on stream-aligned teams so they can focus on delivering value. The CNCF Platforms White Paper frames it similarly: a platform is a curated set of capabilities offered as self-service, with consistent interfaces and documentation. Kubernetes and Terraform are not the platform. They are the most common building blocks for one.
How Platform Engineering Divides Work Between Terraform and Kubernetes
The most important architectural decision is deciding which tool owns which resource. When that boundary is fuzzy, you get two sources of truth fighting each other.
Terraform: The Foundation Layer
Terraform is declarative, plan-based, and state-tracked. That makes it a good fit for resources with long lifecycles and high blast radius:
- Cloud accounts, organizational structure, and IAM baselines
- VPCs, subnets, transit gateways, and DNS zones
- Managed Kubernetes clusters (EKS, AKS, GKE) and their node pool definitions
- Managed data services such as RDS, Cloud SQL, or object storage with strict retention policies
The terraform plan step gives reviewers a concrete diff before anything changes. For foundational infrastructure, that review gate is worth the slower feedback loop.
Kubernetes: The Runtime and Control Plane
Kubernetes runs a continuous reconciliation loop: controllers compare desired state to actual state and correct drift automatically. That makes it a better fit for things that change many times a day:
- Application deployments, services, and ingress rules
- Horizontal scaling and rollouts
- Namespaces, quotas, and RBAC for tenant teams
- Cluster add-ons such as ingress controllers, cert-manager, and observability agents
Where the Boundary Sits
A rule we apply: Terraform creates the cluster and the minimum needed to hand it to GitOps. Everything inside the cluster after that is owned by a GitOps controller. Avoid managing in-cluster application resources through the Terraform Kubernetes provider at scale. It couples application deploys to infrastructure plans and makes state files fragile.
Step-by-Step: Building a Scalable Platform
1. Structure Terraform as Versioned Modules
Write modules once, version them, and consume them by tag. Product teams should call a module, not copy it.
module "orders_network" {
source = "git::https://git.example.com/platform/terraform-modules.git//network?ref=v2.3.0"
name = "orders-prod"
cidr_block = "10.40.0.0/16"
availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"]
enable_nat_gateway = true
tags = {
owner = "team-orders"
environment = "prod"
cost_center = "cc-1142"
}
}
Pinning to ref=v2.3.0 means a module change never silently alters production. Upgrades become deliberate pull requests.
2. Manage State and Environments Deliberately
Remote state with locking is non-negotiable once more than one person or pipeline runs Terraform. Split state by environment and by layer (network, cluster, data) so a mistake in one layer can't corrupt another.
terraform {
required_version = ">= 1.10"
backend "s3" {
bucket = "acme-terraform-state"
key = "prod/network/terraform.tfstate"
region = "us-east-1"
encrypt = true
use_lockfile = true # S3-native locking, Terraform 1.10+
}
}
On older Terraform versions, the S3 backend uses a DynamoDB table for locking. Either way, run plan and apply from CI, not workstations, and require approval on production applies.
3. Bootstrap Clusters into GitOps
Once Terraform creates a cluster, install a GitOps controller (Argo CD or Flux) and point it at a repository. From that point, Git is the source of truth for everything in the cluster.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: platform-addons
namespace: argocd
spec:
project: platform
source:
repoURL: https://git.example.com/platform/cluster-addons.git
targetRevision: main
path: overlays/prod
destination:
server: https://kubernetes.default.svc
syncPolicy:
automated:
prune: true
selfHeal: true
selfHeal: true reverts manual changes, which eliminates the laptop-kubectl drift problem described earlier.
4. Offer Golden Paths, Not Raw Access
A golden path is an opinionated, supported way to do a common task. For a new service, that typically means a template that generates a namespace, resource quotas, network policies, a deployment, CI configuration, and dashboards in one step. Tools like Backstage can expose these templates through a portal, but a well-documented repository template works as a starting point.
The guardrails belong in the template:
apiVersion: v1
kind: ResourceQuota
metadata:
name: team-quota
namespace: team-orders
spec:
hard:
requests.cpu: "20"
requests.memory: 40Gi
limits.cpu: "40"
limits.memory: 80Gi
pods: "100"
Teams can deviate from the golden path, but they take on the support burden when they do. That trade-off keeps the paved road attractive.
5. Enforce Policy as Code
Guardrails only work if they are enforced automatically. Admission controllers such as Kyverno or OPA Gatekeeper reject non-compliant resources at deploy time. On the Terraform side, tools like Checkov, tfsec/Trivy, or OPA with Conftest can scan plans in CI.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-resource-limits
spec:
validationFailureAction: Enforce
rules:
- name: check-container-limits
match:
any:
- resources:
kinds: ["Pod"]
validate:
message: "CPU and memory limits are required."
pattern:
spec:
containers:
- resources:
limits:
memory: "?*"
cpu: "?*"
Roll new policies out in Audit mode first, review the violations, then switch to Enforce. Turning on enforcement cold is a reliable way to break deploys and lose developer trust.
6. Design for Elastic Scale
Scalable cloud architecture requires scaling at two levels:
- Pod level: the Horizontal Pod Autoscaler adjusts replica counts based on CPU, memory, or custom metrics.
- Node level: Cluster Autoscaler or Karpenter adds and removes nodes based on pending pods.
Terraform defines node pool boundaries such as instance families, minimum and maximum sizes, and subnets. Kubernetes autoscalers operate within those boundaries. This preserves the layer split: Terraform sets the envelope, and Kubernetes moves inside it.
Common Pitfalls
- Building a platform no one asked for. Treat the platform as a product. Interview users, prioritize their biggest friction points, and ship incrementally.
- Monolithic Terraform state. A single state file for an entire organization makes every plan slow and every apply risky.
- Too many abstractions too early. Wrapping Kubernetes in a custom DSL before you understand usage patterns creates a second system to maintain.
- No ownership model. Every module, policy, and template needs a named owner and a deprecation process.
Measuring Whether It Works
Measure outcomes, not tool adoption. The DORA research program's four key metrics (deployment frequency, lead time for changes, change failure rate, and time to restore service) are a reasonable baseline. Pair them with developer experience signals: time to first deploy for a new service, number of infrastructure tickets filed, and qualitative survey feedback. Establish a baseline before the platform launches so improvements are visible and defensible.
Where to Start
If you are early in this journey, begin with one painful, high-frequency workflow, such as provisioning a new service environment. Automate it end to end with Terraform modules and a GitOps-managed namespace template, then expand. You can explore how we approach this work across our cloud and platform engineering capabilities, and see how requirements shift in regulated and high-scale environments on our industries page.
FAQ
What is the difference between platform engineering and DevOps?
DevOps is a set of cultural practices and principles around shared ownership of delivery and operations. Platform engineering is one way to put those principles into practice at scale: a dedicated team builds self-service tooling so product teams can own their services without becoming infrastructure experts.
Should Terraform manage resources inside Kubernetes?
Generally, only for bootstrapping, such as installing the GitOps controller. Ongoing in-cluster resources are better managed by a GitOps tool like Argo CD or Flux, which continuously reconciles state. Using Terraform for frequently changing workloads couples application releases to infrastructure plans.
Do we need Backstage to build an internal developer platform?
No. A portal improves discoverability, but the core of an internal developer platform is the automation behind it: modules, templates, pipelines, and policies. Many teams start with documented repository templates and add a portal once usage patterns are clear.
How do we keep Terraform modules from becoming a bottleneck?
Version modules semantically, publish changelogs, accept contributions through pull requests with clear review standards, and automate testing with tools such as terraform test or Terratest. The platform team should own the modules without being the only people allowed to improve them.
How long does it take to stand up a platform?
It depends on your starting point, team size, and compliance requirements. We recommend delivering a thin, usable slice early, such as one golden path for one service type, then iterating based on adoption and feedback rather than planning a single large launch.
Sources
- CNCF TAG App Delivery, Platforms White Paper: tag-app-delivery.cncf.io/whitepapers/platforms
- Skelton, M. and Pais, M., Team Topologies, IT Revolution Press, 2019
- Kubernetes documentation: kubernetes.io/docs
- HashiCorp Terraform documentation, S3 backend: developer.hashicorp.com/terraform/language/backend/s3
- Argo CD documentation: argo-cd.readthedocs.io
- Kyverno documentation: kyverno.io/docs
- DORA research program: dora.dev
