A fifteen-part series
Kubernetes on Bare Metal
From a powered-off server to a self-healing, multi-datacenter platform for serving and training models, built entirely on hardware you own. Each part is a working layer, and one discipline runs through all of them: declare the intent, and let the machinery reconcile the world to match. Every hard thing a cloud does for you and marks up, the control plane, the load balancer, the managed database, the global backbone, the replaced hardware, is owned here instead.
Foundation
A powered-off machine becomes a reconciling cluster.Bootstrap
A powered-off server becomes a control-plane node: Metal3 and Redfish for the hardware, immutable Talos for the OS, Cluster API to orchestrate it.
CNI
The NotReady nodes get a network: Cilium in eBPF, kube-proxy-free, speaking BGP to the rack fabric with WireGuard on the wire.
GitOps
A reconciling brain: Argo CD bootstrapped into itself, with ApplicationSet generators and per-PR preview environments straight from git.
The substrate
What every workload stands on.Secrets
Git holds the reference, OpenBao holds the value: HSM auto-unseal, dynamic engines, and secrets that never sit in a repo.
Storage
Raw NVMe becomes resilient storage: Rook Ceph serving block, file, and object, replicated across racks.
Traffic
North-south and east-west on one Cilium dataplane: a gateway on an owned IP, a sidecar-less mesh, certificates and DNS wired in.
Supply chain
Nothing runs unless the platform built it: Harbor, SBOMs, Cosign signing, and a Kyverno gate at admission.
Operate and defend
Seeing the platform, scaling it, hardening it.Observability
The platform learns to see itself: four signals on the LGTM stack, all on the Ceph object floor, ending in an alert that pages with its own root cause.
Scaling
Reflexes to match the senses: KEDA and VPA for pods, and a Pending pod that turns a spare physical server on.
Runtime security
Defense in depth at runtime: admission policy, sandboxed runtimes, and a Tetragon signal promoted from watching to killing.
The platform as a product
Self-service over everything below.IDP
The platform becomes a product: one Crossplane spec becomes the whole hardened stack, shipped through Backstage, priced and reached without a cloud bill.
Databases
A database is a field, not a project: ten stateful engines behind one Crossplane flag, on Ceph, with backups and credentials that expire.
The frontier
The hard, distinctive parts of owning the metal.GPU
The silicon the models run on: NVIDIA drivers on an immutable OS, sharing a card with MIG, and an InfiniBand fabric you design yourself.
Multi-datacenter
Two sites without a shared fate: federated clusters over a KubeSpan WAN and Cilium ClusterMesh, state replicated not stretched.
Fleet
Cattle all the way down: self-healing to the physical machine, clusters rebuilt from git, a whole fleet run from one desk.
Read them in order for the full build, or jump to a layer. Each part stands on its own but hands off to the next, and the last one closes the loop the first opened: the machinery that brings a machine to life is the machinery that keeps the whole fleet alive. Nothing on this platform is a pet.
Start at Part 1 →