Tech Expert & Vibe Coder

With 15+ years of experience, I specialize in self-hosting, AI automation, and Vibe Coding – building applications using AI-powered tools like Google Antigravity, Dyad, and Cline. From homelabs to enterprise solutions.

Setting up WireGuard mesh networking with dynamic peer discovery using headscale as self-hosted tailscale control server

Why I Set Up Headscale

I run services across multiple locations: a Proxmox cluster at home, a Synology NAS, a Raspberry Pi acting as a DNS server, and a few VMs on cloud providers. For years, I used a mix of SSH tunnels, dynamic DNS, and port forwarding to access these systems remotely. It worked, but it was fragile. Every time my ISP changed my IP or I added a new device, I had to reconfigure something.

I’d heard about Tailscale—WireGuard-based mesh networking with automatic NAT traversal—but I wasn’t comfortable routing all my traffic through a third-party control plane. When I found Headscale, an open-source implementation of the Tailscale coordination server, I decided to try self-hosting it.

My goal was simple: secure, zero-configuration access to my home network from anywhere, without exposing ports or trusting external services with my coordination metadata.

My Setup

I deployed Headscale on a small AWS EC2 instance (t3.micro, Ubuntu 24.04) using Docker. I already had a domain pointing to this instance, so I configured Caddy as a reverse proxy to handle HTTPS.

Docker Compose Configuration

Here’s the docker-compose.yml I used:

services:
  headscale:
    image: headscale/headscale:latest
    container_name: headscale
    restart: always
    ports:
      - "8080:8080"
    volumes:
      - ./config:/etc/headscale
      - ./data:/var/lib/headscale
    command: headscale serve

I created the initial user and generated a reusable pre-authentication key:

docker exec headscale headscale users create vipin
docker exec headscale headscale preauthkeys create --reusable --expiration 720h --user vipin

The pre-auth key allowed me to connect devices without manually approving each one through the web interface.

Firewall Rules

On the EC2 instance, I opened the following ports using ufw:

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 3478/udp
sudo ufw allow 41641/udp

Port 3478 is for STUN, which helps with NAT traversal. Port 41641 is the default WireGuard listen port.

Caddy Configuration

My Caddyfile reverse-proxies Headscale’s API and gRPC endpoints:

headscale.example.com {
    reverse_proxy localhost:8080
}

Caddy automatically handles TLS via Let’s Encrypt, which is critical since Tailscale clients require HTTPS for the coordination server.

Connecting Devices

macOS and Linux

On my MacBook and Ubuntu desktop, I installed the official Tailscale client and pointed it at my Headscale server:

tailscale up --login-server https://headscale.example.com --auth-key [pre-auth-key]

The devices immediately appeared in my Headscale node list. Each one received a 100.x.y.z IP address and could ping the others.

Raspberry Pi (Subnet Router)

I wanted my entire home LAN (192.168.1.0/24) accessible through the mesh. I installed Tailscale on a Raspberry Pi connected to my home network:

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up --login-server https://headscale.example.com 
  --auth-key [pre-auth-key] 
  --advertise-routes=192.168.1.0/24 
  --accept-routes

This advertised my home subnet to the Headscale control server, but the route wasn’t active yet.

Enabling the Subnet Route

Headscale doesn’t automatically approve advertised routes. I had to enable it manually:

docker exec headscale headscale routes list
docker exec headscale headscale routes enable --route 1

After enabling the route, I verified that other devices on the mesh could reach machines on my home network (e.g., my Synology NAS at 192.168.1.50).

IP Forwarding on the Pi

The Raspberry Pi needs to forward packets between the Tailscale interface and my home LAN. I enabled IP forwarding:

echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

I also verified that iptables wasn’t blocking forwarded traffic. By default, Tailscale sets up the necessary rules, but I double-checked with:

sudo iptables -L FORWARD -v

iOS

On my iPhone, I installed the Tailscale app and visited the registration URL provided by Headscale:

https://headscale.example.com/register?key=[device-key]

The device registered successfully, and I could access my home network from my phone over LTE.

What Worked

The mesh network worked immediately. Devices could ping each other using their 100.x.y.z addresses, and DNS resolution worked via MagicDNS (Headscale’s built-in DNS feature). I could SSH into my Proxmox host from my laptop, even when I was on a hotel Wi-Fi network with aggressive NAT.

Subnet routing was the real win. Instead of exposing my Synology NAS to the internet or setting up a traditional VPN, I could access it directly via the Raspberry Pi’s advertised route. This also worked for services like my Home Assistant instance and internal dashboards.

The setup was stable. I haven’t had to touch the Headscale configuration in months. Devices reconnect automatically after reboots or network changes.

What Didn’t Work

Initial Route Confusion

I assumed advertised routes would be automatically enabled. They weren’t. It took me a while to realize I needed to manually approve them using the headscale routes enable command. The error messages weren’t obvious—devices just couldn’t reach the subnet.

DERP Relay Performance

When direct connections failed (rare, but it happens on some restrictive networks), traffic fell back to Headscale’s DERP relay. The default relay configuration uses Tailscale’s public DERP servers, which I wasn’t comfortable with. I considered running my own DERP server but decided the complexity wasn’t worth it for my use case. Most of the time, direct connections work fine.

ACL Management

Headscale’s ACL system is powerful but not intuitive. I wanted to restrict certain devices from accessing specific subnets, but the JSON-based ACL file was harder to debug than I expected. I ended up keeping it simple and relying on device-level firewall rules instead.

No Web UI

Headscale is command-line only. I had to SSH into the EC2 instance every time I wanted to list nodes, approve routes, or generate keys. For a personal setup, this was fine, but it would be tedious at scale.

Key Takeaways

Headscale gives me full control over my mesh network without trusting a third-party service. It’s not as polished as Tailscale’s hosted offering, but the trade-off is worth it for privacy and flexibility.

If you’re comfortable with Docker and basic networking, Headscale is a solid choice. If you need a web UI, automatic ACL management, or don’t want to manage your own infrastructure, Tailscale’s hosted service is probably better.

For my use case—securely accessing my home network from anywhere—Headscale has been rock solid. I no longer worry about dynamic DNS, port forwarding, or VPN configurations. Everything just works.

Leave a Comment

Your email address will not be published. Required fields are marked *