Cloud & Infrastructure8 min read · Oct 26, 2026

What is Platform Engineering? Building an IDP with Kubernetes and Terraform

Techsense DevelopersEngineering

Platform Engineering is the discipline of building and running an Internal Developer Platform (IDP): a curated, self-service layer of tools, infrastructure, and workflows that lets application teams ship software without having to master every underlying system. In practice, a platform team packages Kubernetes clusters, Terraform-managed cloud resources, CI/CD pipelines, and security guardrails into "golden paths," which are paved, supported routes from idea to production. The goal is not to hide infrastructure. The goal is to reduce the cognitive load on developers so they spend their time on product work instead of YAML archaeology.

If your engineers open tickets to get a database, copy pipeline files between repositories, or wait on a central ops team for every namespace, this post is for you. We will define the terms, explain where Kubernetes and Terraform fit, and walk through a concrete starting architecture.

Why Platform Engineering Emerged

The "you build it, you run it" model of DevOps gave teams ownership and speed. It also handed every product team a growing pile of responsibilities: container orchestration, infrastructure as code, observability, secrets, compliance evidence, and cost management. Each team solved these problems slightly differently, and the result was often duplicated effort and inconsistent security posture.

Platform Engineering is a response to that sprawl, not a rejection of DevOps. The book Team Topologies (Skelton and Pais) describes a platform team whose job is to provide internal services that reduce the cognitive load on stream-aligned teams. The CNCF Platforms White Paper frames a platform as an integrated collection of capabilities, presented according to the needs of its users, and emphasizes that it should be treated as a product.

That product framing is the key idea. A platform has customers (your developers), a roadmap, documentation, and success measures. If nobody chooses to use it, it has failed, no matter how elegant the architecture.

What an Internal Developer Platform Actually Contains

An IDP is not a single tool you install. It is a set of layers that work together. We find it helpful to describe it in four parts.

1. The developer interface

This is how developers interact with the platform. It can be a portal (Backstage is the most common open source option), a CLI, a set of repository templates, or simply a well-documented Git workflow. The interface should expose intent ("I need a Postgres database for my service") rather than implementation detail.

2. Orchestration and workload runtime

For most organizations this is Kubernetes. It provides a consistent API for deploying, scaling, and operating workloads across environments and clouds.

3. Infrastructure provisioning

This is where Terraform (or OpenTofu, its open source fork) typically lives. It provisions the clusters themselves, plus networks, IAM, managed databases, queues, and DNS.

4. Guardrails and shared services

Policy enforcement, secrets management, observability, and cost visibility. These are the controls that let you say "yes" to self-service without saying "yes" to risk.

Where Kubernetes Fits in an IDP

Kubernetes is popular as an IDP foundation because it is, at its core, an extensible API with a reconciliation loop. You declare desired state, and controllers converge reality toward it. That model is exactly what a platform needs:

Consistent abstractions across dev, staging, and production.
Multi-tenancy primitives such as namespaces, RBAC, resource quotas, and network policies.
Extensibility through Custom Resource Definitions (CRDs), so the platform team can define higher-level objects.
Policy hooks through admission controllers, which let you enforce standards at deploy time.

That last point matters more than it first appears. Rather than reviewing every manifest by hand, a platform team can encode standards as policy. Here is a Kyverno policy that rejects pods without resource requests and memory limits:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-requests-limits
spec:
  validationFailureAction: Enforce
  rules:
    - name: validate-resources
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message: "CPU and memory requests, and a memory limit, are required."
        pattern:
          spec:
            containers:
              - resources:
                  requests:
                    cpu: "?*"
                    memory: "?*"
                  limits:
                    memory: "?*"

A developer who forgets limits gets an immediate, explicit error at deploy time instead of a noisy-neighbor incident three weeks later. That is DevOps efficiency in a concrete form: feedback moves left, and the platform team stops being a review bottleneck.

Where Terraform Fits in an IDP

Kubernetes manages workloads well, but something has to create the clusters, VPCs, IAM roles, and managed cloud services underneath. Terraform is well suited to this because it is declarative, provider-agnostic, and has a mature module ecosystem.

The most important Terraform pattern for an IDP is the opinionated module. Instead of letting every team write raw cloud resources, the platform team publishes modules that encode your defaults: tagging, encryption, network placement, and least-privilege access. Teams consume them with a handful of inputs.

Here is a simplified module that onboards a team onto a shared cluster:

# modules/team-namespace/main.tf
variable "team"         { type = string }
variable "cpu_quota"    { type = string, default = "8" }
variable "memory_quota" { type = string, default = "16Gi" }
variable "admin_group"  { type = string }

resource "kubernetes_namespace_v1" "this" {
  metadata {
    name   = var.team
    labels = { "platform.example.com/team" = var.team }
  }
}

resource "kubernetes_resource_quota_v1" "this" {
  metadata {
    name      = "${var.team}-quota"
    namespace = kubernetes_namespace_v1.this.metadata[0].name
  }
  spec {
    hard = {
      "requests.cpu"    = var.cpu_quota
      "requests.memory" = var.memory_quota
    }
  }
}

resource "kubernetes_role_binding_v1" "admins" {
  metadata {
    name      = "${var.team}-admins"
    namespace = kubernetes_namespace_v1.this.metadata[0].name
  }
  role_ref {
    api_group = "rbac.authorization.k8s.io"
    kind      = "ClusterRole"
    name      = "admin"
  }
  subject {
    kind      = "Group"
    name      = var.admin_group
    api_group = "rbac.authorization.k8s.io"
  }
}

Consuming it is short and reviewable:

module "team_payments" {
  source       = "./modules/team-namespace"
  team         = "payments"
  admin_group  = "eng-payments"
  memory_quota = "32Gi"
}

Onboarding a new team becomes a small pull request rather than a ticket queue. The module is versioned, tested, and auditable.

Terraform vs. Kubernetes-native provisioning

A common design question is whether cloud resources such as databases should be provisioned by Terraform pipelines or by Kubernetes controllers like Crossplane or the cloud vendors' operators. Both work. Our general guidance:

Use Terraform for foundational, slower-changing infrastructure: accounts, networks, clusters, shared IAM.
Consider Kubernetes-native controllers for per-application resources that developers request frequently, so the request lives next to the workload and benefits from continuous reconciliation.

Pick one primary path for each resource type and document it. Mixing both for the same resource creates drift and confusion.

A Reference Architecture for Your First IDP

Here is a pragmatic starting shape. It deliberately avoids building everything at once.

Foundation (Terraform): cloud accounts, networking, and one or more managed Kubernetes clusters, with state stored remotely and locked.
Cluster baseline (GitOps): Argo CD or Flux installs ingress, cert-manager, external secrets, policy engine, and observability agents from a platform repository.
Team onboarding (Terraform modules): namespaces, quotas, RBAC, and cloud IAM bindings per team, as shown above.
Golden path templates: a service template that includes a Dockerfile, CI pipeline, Kubernetes manifests or Helm chart, and a catalog entry.
Developer interface: start with templates and docs. Add a portal when the number of services makes discovery genuinely hard.

A catalog entry for Backstage is small and gives every service an owner and a home:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: payments-api
  description: Handles card authorization and settlement
  annotations:
    github.com/project-slug: example-org/payments-api
spec:
  type: service
  lifecycle: production
  owner: team-payments
  system: payments

Common Pitfalls

We see the same mistakes repeatedly in platform initiatives, regardless of company size:

Building a portal first. A portal on top of inconsistent infrastructure is a pretty front door to a messy house. Standardize the paths, then surface them.
Mandating instead of attracting. If the golden path is harder than the workaround, teams will route around it. Make the paved road the easiest road.
Over-abstracting. Hiding Kubernetes completely often backfires during incidents. Provide sensible defaults and leave escape hatches for teams that need them.
No product ownership. Without a roadmap and user feedback loop, a platform becomes a collection of scripts that one person understands.
Measuring activity instead of outcomes. Track things developers feel, such as time to first deploy for a new service and lead time for changes. The DORA research program offers a well-established set of delivery metrics to start from.

Getting Started

Begin with one painful, frequent workflow, such as provisioning a new service or onboarding a team, and turn it into a paved path. Interview developers before writing code. Ship a thin version, measure it, and iterate.

If you are weighing how to staff or sequence this work, our cloud and platform engineering capabilities page outlines how we approach these engagements, and our industry-specific perspectives cover how regulatory and scale requirements shape platform design in different sectors.

Sources and Further Reading

Skelton, M. and Pais, M. Team Topologies: Organizing Business and Technology Teams for Fast Flow. IT Revolution, 2019.
CNCF TAG App Delivery. CNCF Platforms White Paper. tag-app-delivery.cncf.io.
Kubernetes documentation: Multi-tenancy, Resource Quotas, and RBAC. kubernetes.io/docs.
HashiCorp Terraform documentation: Modules and the Kubernetes provider. developer.hashicorp.com/terraform.
Kyverno documentation: Validation policies. kyverno.io/docs.
Backstage documentation: Software Catalog. backstage.io/docs.
DORA research program. dora.dev.

FAQ

Is Platform Engineering replacing DevOps?

No. Platform Engineering builds on DevOps principles such as automation, shared ownership, and fast feedback. It addresses a scaling problem: when every team owns the entire toolchain, cognitive load and inconsistency grow. A platform team absorbs common concerns so product teams can keep the "you build it, you run it" model without reinventing infrastructure.

Do I need Kubernetes to build an Internal Developer Platform?

Not strictly. An IDP can be built on serverless services, PaaS offerings, or virtual machines. Kubernetes is a common choice because its declarative API, extensibility through CRDs, and multi-tenancy primitives map well to self-service and policy enforcement. Choose it when those properties solve real problems for your teams.

Should we use Terraform or Crossplane for an IDP?

Many platforms use both. Terraform is well suited to foundational infrastructure like networks, accounts, and clusters. Kubernetes-native tools like Crossplane suit per-application resources that developers request often. The important thing is to choose one primary tool per resource type and document the decision.

How small can a platform team be?

There is no fixed number. What matters is that someone owns the platform as a product, with a roadmap and a feedback loop with developers. Start with a narrow scope, such as one golden path, so the team can support what it ships.

How do we know if our IDP is working?

Look at developer outcomes rather than tool adoption alone. Useful signals include time to first production deploy for a new service, lead time for changes, change failure rate, and qualitative feedback from developer surveys. If teams voluntarily choose the golden path, that is a strong sign it is delivering value.