Configuring Proxmox LXC containers with systemd-nspawn for running Docker workloads with better resource isolation than VMs

Why I Started Looking at systemd-nspawn for Docker Workloads

I run a lot of Docker containers on Proxmox. For years, my setup was straightforward: one or two VMs running Docker Engine, each handling 10-20 containers. It worked, but it always felt wasteful. Each VM consumed 2-4GB of RAM just sitting idle, and I had to maintain separate OS instances, updates, and kernels.

LXC containers seemed like the obvious alternative. They're lightweight, boot fast, and integrate cleanly with Proxmox. But running Docker inside LXC has always been awkward. You need privileged containers or specific kernel module workarounds, and even then, networking gets messy. I tried it a few times and always went back to VMs.

Then Proxmox 9.1 added native OCI container support, which uses systemd-nspawn under the hood. I didn't fully understand what that meant at first, but after digging into how it actually works, I realized systemd-nspawn could give me something better than both VMs and traditional LXC: proper resource isolation without the VM overhead, and a cleaner way to run Docker workloads without hacking privileged containers.

What systemd-nspawn Actually Does

systemd-nspawn is a container runtime built into systemd. It's not Docker, and it's not LXC. It sits somewhere in between. It creates isolated namespaces for processes, filesystems, and networks, but it doesn't require a separate init system or a full VM kernel.

When Proxmox pulls an OCI image and runs it as a container, it's actually wrapping that image in an LXC container and using systemd-nspawn to execute it. This matters because systemd-nspawn gives you better control over cgroups, resource limits, and filesystem isolation than running Docker directly on the host or inside a traditional LXC container.

What I found interesting is that you can also manually configure LXC containers to use systemd-nspawn for running Docker workloads. This isn't the same as Proxmox's new OCI feature—it's more hands-on—but it gives you finer control over how containers are isolated and how resources are allocated.

My Setup: LXC with systemd-nspawn for Docker

I started with a clean Debian 12 LXC container on Proxmox. I wanted to test whether I could run Docker inside it using systemd-nspawn for better isolation, without making the container fully privileged.

Step 1: Creating the LXC Container

I created an unprivileged LXC container with these settings:

• 2 CPU cores
• 2GB RAM
• 20GB root disk on ZFS
• Nested virtualization enabled
• Keyctl and nesting features enabled in the container config

In the Proxmox host, I edited /etc/pve/lxc/[container-id].conf and added:

features: keyctl=1,nesting=1
lxc.apparmor.profile: unconfined
lxc.cgroup2.devices.allow: a
lxc.cap.drop:
lxc.mount.auto: proc:rw sys:rw cgroup:rw

This is not a fully privileged container, but it does relax some restrictions. The key difference from a standard privileged LXC setup is that I'm not giving it full root access to the host—I'm just allowing it to manage its own cgroups and namespaces.

Step 2: Installing Docker Inside the Container

Inside the LXC container, I installed Docker using the official Debian repository:

apt update
apt install -y ca-certificates curl gnupg
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg

echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null

apt update
apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Docker started without errors, which was already better than my previous LXC attempts.

Step 3: Configuring systemd-nspawn for Isolation

By default, Docker containers inside the LXC share the same cgroup namespace. This means resource limits aren't as strict as I wanted. To improve isolation, I configured systemd-nspawn to wrap Docker containers in their own namespaces.

I created a systemd-nspawn configuration file at /etc/systemd/nspawn/docker.nspawn:

[Exec]
Boot=yes
PrivateUsers=yes

[Files]
PrivateUsersChown=yes
Bind=/var/run/docker.sock

[Network]
Private=no

This tells systemd-nspawn to:

• Use user namespaces for better privilege separation
• Allow Docker socket access
• Keep network access shared (since I'm using Proxmox's bridge networking)

I restarted the container and verified that Docker was still running. It was.

What Worked

Once everything was configured, I spun up a few test containers: Nginx, a small Python API, and a PostgreSQL database. All of them started without issues.

What I noticed immediately:

• Resource limits actually worked. When I set CPU and memory limits on the LXC container, Docker respected them without leaking resources to the host.
• Filesystem isolation was cleaner. Docker's overlay filesystem didn't interfere with the host's ZFS pool.
• Networking was simpler than running Docker in a VM. I could assign the LXC container a static IP on my VLAN, and all Docker containers inside it used that IP with port mapping.
• The LXC container used about 400MB of RAM at idle, compared to 2GB for a full VM running Docker.

I also tested backups. Proxmox's built-in backup tool captured the entire LXC container, including all Docker volumes. Restoring it to a different node worked without any manual reconfiguration.

What Didn't Work

Not everything was smooth.

Docker Compose Networking Issues

When I tried to run a Docker Compose stack with custom bridge networks, I hit permission errors. Docker couldn't create new network interfaces inside the LXC container, even with nesting enabled.

I worked around this by using host networking for most containers, but that's not ideal if you want full network isolation between services.

Limited Support for Advanced Docker Features

Some Docker features didn't work at all:

• Swarm mode failed to initialize because the LXC container couldn't manage its own routing tables.
• Macvlan networking was completely broken. I couldn't assign containers their own IPs on my physical network.
• Kubernetes (k3s) wouldn't run. It needs more kernel access than an LXC container can provide, even with systemd-nspawn.

If you need any of these features, a full VM is still the only option.

No Clear Update Path for OCI Images

Proxmox 9.1's native OCI support is new, and there's no built-in way to update containers pulled from Docker Hub. You have to manually delete and recreate them. This is fine for testing, but not practical for production workloads.

My manual systemd-nspawn setup doesn't have this problem because I'm just running Docker normally, but it's worth noting if you're considering Proxmox's built-in OCI feature.

Key Takeaways

Using systemd-nspawn to run Docker workloads inside LXC containers is a middle ground between full VMs and native LXC. It works well for lightweight Docker stacks that don't need advanced networking or orchestration.

Here's when it makes sense:

• You're running 5-15 containers that don't need custom bridge networks or Swarm.
• You want better resource isolation than a shared Docker VM, but don't want the overhead of multiple VMs.
• You're okay with some manual configuration and occasional troubleshooting.

Here's when it doesn't:

• You need Kubernetes, Swarm, or complex multi-host networking.
• You want a fully supported, zero-maintenance setup.
• You're running production workloads that can't tolerate experimental configurations.

For my home lab, this setup replaced two Docker VMs and freed up about 3GB of RAM. I'm still running one VM for stacks that need advanced networking, but most of my single-service containers now run in LXC with systemd-nspawn.

It's not perfect, but it's the most resource-efficient way I've found to run Docker on Proxmox without sacrificing too much flexibility.