Falco to Tetragon: The Migration We Deferred
Moving our dataplane to Cilium made Tetragon the obvious next step for runtime security. Then a feasibility pass showed why a clean cutover was not possible yet. Here is what we found, and why I still want to make the move.
When we moved our dataplane to Cilium eBPF, the natural next question was runtime security. We run Falco today, and Tetragon is the runtime-security tool of the Cilium ecosystem, so the migration looked obvious and nearly free. It was not. A pre-migration feasibility pass turned up a gap that made a clean cutover impossible, so we deferred it, on purpose. This is that story, and why I still want to make the move.
Why we wanted to leave Falco
Falco has served us well as runtime security monitoring: it watches syscalls, matches them against rules, and alerts when something looks wrong. But out of the box it is audit-only. It can tell you a shell just spawned in a production container; it cannot stop it. Tetragon can. It is built on the same eBPF foundation as the dataplane we had just adopted, it filters and acts in-kernel, and it can enforce: kill the process, override the syscall, block the action, in real time.
To be fair to Falco, audit-only is not quite the whole story. You can bolt enforcement on: Falco forwards its events to falcosidekick, which forwards them to falco-talon, which runs the response. It works, and I am glad the option exists. But it is two extra components and a round-trip out to userspace and back, where Tetragon decides and acts in one place, inside the kernel.
So the pitch to ourselves was simple. Consolidate on the Cilium ecosystem, get enforcement instead of just detection, and get eBPF's low overhead at scale. Here is how the two actually compare:
Two runtime-security tools, two different philosophies. Pick a dimension.
A security camera. It watches and raises the alarm after the fact: it alerts that a file in /etc was modified, but the change is already made.
A bouncer. It stops the action before it completes: it kills the process mid-write, so the file in /etc is never touched.
The model is different: rules versus hook points
The first thing you learn porting Falco to Tetragon is that you are not translating rules, you are rethinking them. A Falco rule is a condition over syscall events or plugin fields, with an output string and a priority. A Tetragon TracingPolicy is lower-level: you choose which kernel hook points to attach to, which fields to extract, and which actions to take, enforcement included. Here is the same intent, no interactive shell in production, expressed both ways:
# Falco: a rule matching the exec syscall. Detect only.
- rule: Shell in a container
condition: spawned_process and container and shell_procs
output: "Shell spawned (proc=%proc.name container=%container.name)"
priority: WARNING
# Tetragon: the same intent as a TracingPolicy, at the kernel hook, with enforcement.
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: no-shell-in-prod
spec:
kprobes:
- call: sys_execve
syscall: true
selectors:
- matchBinaries:
- operator: In
values: ["/bin/sh", "/bin/bash"]
matchActions:
- action: Sigkill # enforce, not just alertChoosing the hook point
That last part is the real learning curve. Falco gives you a rule language that abstracts the kernel away. Tetragon gives you the kernel. Every TracingPolicy begins with a decision Falco never made you make: which hook fits the intent, a kprobe, a tracepoint, a uprobe, or an LSM hook. Each has different reach, stability, and enforcement power.
A TracingPolicy starts with a choice Falco never made you make: which kernel hook fits the intent. Click one.
- Attach to almost any kernel function on the fly, which gives you maximum reach into what the kernel is doing.
- The catch: function names and signatures are not a stable API, so a policy written against one kernel can break on an upgrade or a different distro.
This is the learning curve. Falco hands you a rule language; Tetragon hands you the kernel. The power is real, but you have to know where in the kernel your security intent actually lives.
Where it fell apart: not every rule maps
The wall we hit in the feasibility discovery was not the learning curve. It was coverage. Tetragon sees the kernel, and it enriches kernel events with Kubernetes and container context, which is excellent. But it does not ingest Kubernetes audit logs, control-plane activity, or cloud provider logs, and a real chunk of our Falco rules did.
Sort the detections by where their signal comes from and the problem is obvious:
Every Falco rule had to become a Tetragon TracingPolicy, or stay behind. Click a detection to see which. This is what the feasibility discovery turned up.
A shell spawned inside a container is a kernel event. A TracingPolicy on execve catches it, and can kill it, not just log it.
The kernel-visible half of our detections ports cleanly. The audit and control-plane half does not, because there is no kernel hook for something that happens in the API server. A clean cutover would have quietly dropped those detections, so we could not just flip the switch.
Why we deferred, and why we will still do it
Put together, a clean cutover would have done one of two bad things: quietly dropped the audit and control-plane detections, or forced us to run Falco and Tetragon side by side indefinitely, which is more moving parts, not fewer. Neither was worth rushing into during a migration that was already swapping the CNI and the mesh and jumping six Kubernetes versions. So we deferred it, deliberately, rather than firefight it.
But I still want to make the move, and I think it is the right one. Enforcement instead of audit-only, one ecosystem instead of two, and genuine kernel-level visibility are real gains. The plan when we do it is to treat it as a modernization rather than a like-for-like port: translate the kernel-visible rules to TracingPolicies, choose the hook point per intent, and keep a separate, purpose-built pipeline for the audit and cloud sources that were never a kernel concern in the first place. Done right, a migration like this is an upgrade of how you express security intent, not a search-and-replace.