Building a container escape detection system with falco and ntfy alerts for runtime security monitoring

Why I Built Container Escape Detection

I run multiple Docker containers on my Proxmox homelab, some of them exposed to the internet through Cloudflare tunnels. One day, I realized I had no visibility into what was happening inside those containers at runtime. If something broke out of a container or started doing weird syscalls, I'd only know after damage was done.

I needed a way to detect suspicious behavior as it happened, not after logs piled up or services crashed. That's when I started looking at Falco.

What Falco Actually Does

Falco watches kernel-level system calls in real time. It doesn't scan files or check signatures—it monitors behavior. If a container suddenly spawns a shell, writes to /etc, or makes an unexpected network connection, Falco catches it.

It uses eBPF (or a kernel module fallback) to hook into syscalls without heavy overhead. The rules are written in YAML, which made them easy for me to read and modify.

I'm not running a Kubernetes cluster—just plain Docker on Proxmox. Falco still works fine for this setup.

My Setup

I installed Falco directly on the Proxmox host (Debian-based). This way, it monitors all containers running on that node without needing to be inside each container.

Installation was straightforward:

curl -s https://falco.org/repo/falcosecurity-packages.asc | apt-key add -
echo "deb https://download.falco.org/packages/deb stable main" | tee /etc/apt/sources.list.d/falcosecurity.list
apt update
apt install -y falco

Falco runs as a systemd service. I left the default driver as eBPF since my kernel version supported it.

Default Rules

Out of the box, Falco ships with a comprehensive rule set in /etc/falco/falco_rules.yaml. These cover things like:

• Shell execution inside containers
• Writing to sensitive directories like /etc or /bin
• Privilege escalation attempts
• Unexpected network connections

I ran it with defaults first to see what triggered. Turns out, some of my legitimate containers (like my n8n instance) occasionally spawn shells for workflows. That meant I needed custom rules with exceptions.

Writing My First Custom Rule

I wanted to detect if any container tried to execute nc (netcat) or wget, since those are common in reverse shells or data exfiltration.

Here's what I wrote in /etc/falco/falco_rules.local.yaml:

- rule: Suspicious binary execution in container
  desc: Detect execution of potentially dangerous binaries
  condition: >
    spawned_process and
    container and
    proc.name in (nc, ncat, wget, curl) and
    not container.image.repository in (n8n/n8n, my-trusted-image)
  output: >
    Suspicious binary executed (user=%user.name command=%proc.cmdline 
    container=%container.id image=%container.image.repository)
  priority: WARNING
  tags: [container, network, security]

This rule triggers when one of those binaries runs inside a container, except for containers I explicitly trust (like n8n, which uses curl for HTTP requests).

What I Learned About Rule Syntax

Falco rules use a condition language that references syscall fields. Key ones I used:

• spawned_process – matches execve syscalls
• container – filters events to containers only
• proc.name – the binary being executed
• container.image.repository – the Docker image name
• user.name – the user running the process

The not clause lets me add exceptions. Without it, I'd get alerts every time n8n made an HTTP request.

Detecting Container Escapes

The real goal was catching container escape attempts. These often involve:

• Mounting host filesystems
• Accessing /proc or /sys in unusual ways
• Spawning processes with elevated capabilities

I added this rule:

- rule: Container escape attempt detected
  desc: Detect potential container breakout behavior
  condition: >
    spawned_process and
    container and
    (proc.name in (docker, runc, ctr) or
     fd.name startswith /proc/sys/kernel or
     fd.name startswith /host)
  output: >
    Possible container escape (user=%user.name command=%proc.cmdline 
    container=%container.id file=%fd.name)
  priority: CRITICAL
  tags: [container, escape, runtime-security]

This triggers if a container tries to run Docker commands (which shouldn't happen unless it's Docker-in-Docker), access kernel tunables, or touch paths mounted from the host.

I haven't seen this fire in production yet, which is good. But I tested it by manually running docker ps from inside a container with the Docker socket mounted—it caught it immediately.

Sending Alerts to ntfy

Falco can output alerts in multiple ways: syslog, JSON files, or HTTP webhooks. I use ntfy for push notifications, so I configured Falco to send alerts there.

In /etc/falco/falco.yaml, I enabled the HTTP output:

http_output:
  enabled: true
  url: "https://ntfy.sh/my-falco-alerts"
  user_agent: "falco-ntfy"

Now when a rule triggers, Falco POSTs the alert to ntfy, and I get a notification on my phone.

Why ntfy Works for This

ntfy is dead simple—no API keys, no accounts, just a URL. I can