Setting up Traefik with PROXY protocol v2 to preserve real client IPs behind Tailscale Funnel and Cloudflare

Why I Worked on This

I run several self-hosted services behind Traefik on my Proxmox cluster. Some of these services are exposed through Tailscale Funnel, and others through Cloudflare's proxy. The problem I kept hitting was that my application logs and rate limiting were completely broken because Traefik was seeing the proxy IP instead of the actual visitor's IP.

This wasn't just an academic problem. My rate limiting middleware was useless because every request appeared to come from the same Cloudflare or Tailscale IP. Security logs were meaningless. I needed a way to preserve the real client IP through multiple layers of proxies without opening myself up to IP spoofing attacks.

My Real Setup

Here's what I'm actually running:

• Traefik v3.0 in Docker on Proxmox
• Some services exposed via Tailscale Funnel (which adds its own proxy layer)
• Other services behind Cloudflare (which also proxies and adds headers)
• Rate limiting and access control that depend on accurate client IPs

The traffic flow looks like this: Client → Cloudflare/Tailscale → Traefik → Backend Service

Each proxy layer adds the previous IP to the X-Forwarded-For header. By the time the request reaches my backend, the header chain might look like:

X-Forwarded-For: 203.0.113.45, 104.16.0.1, 172.19.0.2

Where 203.0.113.45 is the real client, 104.16.0.1 is Cloudflare's IP, and 172.19.0.2 is my internal Docker network IP.

The Core Problem: PROXY Protocol vs X-Forwarded-For

I initially tried using just the X-Forwarded-For header. The problem is that this header is trivially spoofable. Anyone can send:

curl -H "X-Forwarded-For: 1.2.3.4" https://mysite.com

And if Traefik blindly trusts that header, my rate limiting sees 1.2.3.4 instead of the real IP. This is a security hole.

PROXY protocol v2 solves this by having the trusted proxy (Cloudflare or Tailscale) send the real client IP in a binary protocol header that cannot be spoofed by the client. The key is that Traefik must be configured to only accept PROXY protocol from trusted sources.

Enabling PROXY Protocol on Traefik

In my Traefik static configuration, I added:

--entryPoints.web.proxyProtocol.trustedIPs=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22
--entryPoints.web.forwardedHeaders.trustedIPs=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22
--entryPoints.websecure.proxyProtocol.trustedIPs=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22
--entryPoints.websecure.forwardedHeaders.trustedIPs=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22

These are Cloudflare's IP ranges. For Tailscale Funnel, I added my Tailscale subnet (100.64.0.0/10).

The critical part: I did NOT use proxyProtocol.insecure or forwardedHeaders.insecure. Those flags tell Traefik to trust ANY source, which defeats the entire security model.

The Depth Problem

Even with PROXY protocol enabled, I still had an issue. When a request comes through Cloudflare, the X-Forwarded-For header might contain:

X-Forwarded-For: [real client IP], [cloudflare IP], [internal IP]

Traefik's built-in behavior was to take the last IP before the trusted proxy, but I needed more control. If someone sent a spoofed X-Forwarded-For header before hitting Cloudflare, I needed to ignore those spoofed entries.

This is where forwardedForDepth comes in. The idea is to count backwards from the trusted proxy to find the real client IP, skipping any spoofed entries.

My Actual Depth Configuration

For Cloudflare: forwardedForDepth: 1 (the IP immediately before Cloudflare's IP is the real client)

For Tailscale Funnel: forwardedForDepth: 1 (same logic)

I did not need depth 2 or higher because I only have one trusted proxy layer in front of Traefik. If I had a setup like Client → CDN → Load Balancer → Traefik, I would need depth 2.

What Worked

After configuring PROXY protocol with trusted IPs and setting the correct depth, my logs started showing real client IPs. Rate limiting worked correctly. I could see actual geographic distribution of traffic.

The key steps that worked:

1. Enable PROXY protocol on Traefik entrypoints with explicit trusted IP ranges
2. Configure forwardedHeaders with the same trusted ranges
3. Set forwardedForDepth to match my proxy topology
4. Verify with curl tests that spoofed headers are ignored

I tested this by sending requests with fake X-Forwarded-For headers directly to my Traefik instance (bypassing Cloudflare). Traefik correctly ignored the spoofed header because the source IP wasn't in the trusted range.

What Didn't Work

My first attempt used proxyProtocol.insecure=true. This made Traefik accept PROXY protocol from any source, which meant a malicious client could send a crafted PROXY protocol header and spoof their IP. This is worse than not using PROXY protocol at all.

I also tried using Traefik's built-in IP whitelist middleware to trust X-Forwarded-For, but this doesn't protect against spoofing if the request comes from a trusted IP range (like when another service in my Docker network makes a request).

Another failed approach: trying to use multiple middleware chains to strip and re-add headers. This created race conditions and unpredictable behavior. The solution needs to be at the entrypoint level, not in middleware.

The Plugin Approach (What I Didn't Use)

The research content mentions a custom Traefik plugin for handling real IPs. I looked at this approach but decided against it for my setup. Here's why:

The plugin requires:

• Building and maintaining custom Go code
• Setting up init containers to clone the plugin
• Managing plugin versions and updates
• Additional complexity in the Traefik deployment

For my use case, Traefik's built-in PROXY protocol support with proper trusted IP configuration was sufficient. The plugin would be useful if I needed custom logic for determining the real IP, but I don't.

I did review the plugin code to understand the forwardedForDepth concept better, which helped me configure Traefik's native settings correctly.

Cloudflare-Specific Configuration

On the Cloudflare side, I had to enable "True-Client-IP Header" in the Cloudflare dashboard under Network settings. Without this, Cloudflare doesn't send the PROXY protocol header.

I also had to ensure my DNS records were set to "Proxied" (orange cloud) rather than "DNS only". This seems obvious but I initially had one subdomain set to DNS-only and couldn't figure out why it wasn't working.

Tailscale Funnel Considerations

Tailscale Funnel automatically uses PROXY protocol v2 when forwarding traffic. I didn't need to configure anything on the Tailscale side. However, I did need to add my Tailscale network's IP range to Traefik's trusted IPs.

One gotcha: Tailscale IPs are in the 100.64.0.0/10 range (CGNAT space). Make sure your firewall rules don't block this range if you're running strict ingress rules.

Testing and Verification

To verify everything was working, I used a simple whoami container behind Traefik and checked the logs:

docker run -d --name whoami traefik/whoami

Then I made requests through Cloudflare and checked the X-Real-IP header that Traefik sets. It correctly showed my actual client IP, not Cloudflare's IP.

I also tested spoofing attempts:

curl -H "X-Forwarded-For: 1.2.3.4" https://mysite.com/whoami

The logs showed my real IP, not 1.2.3.4. This confirmed that Traefik was correctly ignoring spoofed headers.

Key Takeaways

• PROXY protocol v2 is the correct solution for preserving client IPs through trusted proxies
• Never use insecure mode in production - always specify trusted IP ranges explicitly
• The forwardedForDepth setting must match your actual proxy topology
• Test with spoofed headers to verify your configuration is secure
• Traefik's built-in support is sufficient for most setups - custom plugins add complexity
• Both Cloudflare and Tailscale support PROXY protocol, but require different configuration steps

Current Limitations

This setup works well but has some constraints:

• If Cloudflare's IP ranges change, I need to manually update Traefik's configuration
• Services that bypass Cloudflare/Tailscale and hit Traefik directly won't have PROXY protocol headers
• The configuration is split between Traefik, Cloudflare dashboard, and Tailscale settings - no single source of truth
• Debugging requires checking logs at multiple layers to trace where IPs are being set or modified

I'm comfortable with these trade-offs for my setup, but they're worth considering if you're designing a more complex infrastructure.