Tech Expert & Vibe Coder

With 14+ years of experience, I specialize in self-hosting, AI automation, and Vibe Coding – building applications using AI-powered tools like Google Antigravity, Dyad, and Cline. From homelabs to enterprise solutions.

Troubleshooting Nginx reverse proxy websocket timeouts for Home Assistant and Frigate NVR after upgrading to HTTP/3 QUIC

Why I Had to Fix This

I run Home Assistant and Frigate NVR behind an Nginx reverse proxy on my Proxmox host. Both services rely heavily on WebSockets—Home Assistant for real-time dashboard updates and Frigate for live camera streams. Everything worked fine for months.

Then I upgraded Nginx to enable HTTP/3 with QUIC support. I wanted faster initial connections and better performance on mobile devices checking camera feeds remotely. The upgrade went smoothly, SSL labs showed HTTP/3 working, static pages loaded fine.

But within hours, I noticed Home Assistant's dashboard would freeze after a few minutes. Frigate's live view would show "Connecting..." and never recover. Both services were running normally—I could access them directly via their ports. The problem was clearly in the proxy layer.

My Actual Setup

I'm running Nginx 1.25.3 compiled with --with-http_v3_module on Debian 12. The reverse proxy handles:

  • Home Assistant on homeassistant.local:8123
  • Frigate on frigate.local:5000
  • SSL termination with Let's Encrypt certificates
  • External access via Cloudflare DNS (proxied off for these subdomains)

My original working config for Home Assistant looked like this:

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    listen 443 quic reuseport;
    listen [::]:443 quic reuseport;
    
    server_name ha.example.com;
    
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    
    add_header Alt-Svc 'h3=":443"; ma=86400';
    
    location / {
        proxy_pass http://homeassistant.local:8123;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Frigate had nearly identical config. After the HTTP/3 upgrade, these configs stayed the same—I only added the QUIC listen directives and Alt-Svc header.

What Failed First

The symptoms were consistent:

  • WebSocket connections would establish successfully
  • Data would flow for 30-90 seconds
  • Then silence—no errors in browser console, just frozen state
  • Nginx access logs showed the connection still "open" with a 101 status
  • Nginx error logs showed nothing useful

I checked Home Assistant logs—no connection drops reported. Frigate showed the same. The services thought the connection was fine. But the browser never received updates.

My first assumption was a timeout setting. I added these to the location block:

proxy_read_timeout 3600s;
proxy_send_timeout 3600s;

No change. WebSockets still died after the same interval.

The HTTP/3 Connection Problem

I spent an hour reading Nginx HTTP/3 documentation and found something I'd missed: QUIC connections have different timeout behaviors than TCP. The Alt-Svc header I added was telling browsers to prefer HTTP/3, which they did.

But here's what I didn't understand initially—when a browser upgrades to WebSocket over HTTP/3, the connection can't actually use QUIC for the WebSocket itself. WebSocket requires a reliable ordered stream, which HTTP/3 provides through QUIC streams, but the upgrade mechanism is different.

The real issue: my Nginx was accepting HTTP/3 connections, browsers were attempting WebSocket upgrades over those connections, and something in the negotiation was breaking silently.

What Actually Fixed It

I needed to force WebSocket connections to use HTTP/2 or HTTP/1.1, not HTTP/3. The solution was to disable HTTP/3 specifically for the WebSocket upgrade path.

Here's what worked:

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    listen 443 quic reuseport;
    listen [::]:443 quic reuseport;
    
    server_name ha.example.com;
    
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    
    # Only advertise HTTP/3 for non-WebSocket paths
    add_header Alt-Svc 'h3=":443"; ma=86400' always;
    
    location / {
        proxy_pass http://homeassistant.local:8123;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Force HTTP/2 for WebSocket upgrades
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        
        # Longer timeouts for persistent connections
        proxy_read_timeout 86400s;
        proxy_send_timeout 86400s;
        
        # Disable buffering for real-time data
        proxy_buffering off;
    }
}

I also added this to the http block:

map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}

This ensures the Connection header is only set to "upgrade" when an actual upgrade request comes in.

But this alone wasn't enough. The real fix required one more change—I had to remove QUIC from the listen directive for these specific services:

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    # Removed QUIC listen directives entirely
    
    server_name ha.example.com;
    
    # No Alt-Svc header at all
    
    location / {
        # Same proxy settings as above
    }
}

This completely disables HTTP/3 for Home Assistant and Frigate. Static assets load over HTTP/2, WebSocket connections work reliably.

Why This Happened

After fixing it, I dug deeper into the HTTP/3 spec and Nginx's implementation. The problem is that WebSocket over HTTP/3 uses a different mechanism called WebTransport, which wasn't what Home Assistant or Frigate were expecting.

When a browser connects via HTTP/3 and tries to upgrade to WebSocket using the standard HTTP/1.1 upgrade mechanism, Nginx handles it but the connection characteristics change. QUIC's connection migration and packet loss recovery work differently than TCP, and this was causing silent failures in long-lived connections.

Home Assistant's WebSocket implementation expects standard TCP behavior. When packets were lost or reordered in QUIC streams, the application layer didn't handle it correctly because it wasn't designed for QUIC's recovery mechanisms.

What I Learned About HTTP/3 and WebSockets

HTTP/3 is great for request-response patterns and short-lived connections. It's not yet reliable for WebSocket upgrades in all applications, especially those not explicitly designed for it.

The Nginx documentation mentions this briefly but doesn't make it obvious. The assumption is that if you're enabling HTTP/3, your applications support it fully. That's not true for most self-hosted services.

I now run HTTP/3 on my public-facing static sites and documentation. For Home Assistant, Frigate, and any other service using WebSockets, I keep them on HTTP/2 only.

Current Working Configuration

I split my Nginx config into two categories:

HTTP/3 enabled (static content, APIs):

server {
    listen 443 ssl http2;
    listen 443 quic reuseport;
    add_header Alt-Svc 'h3=":443"; ma=86400';
    # Standard proxy settings
}

HTTP/2 only (WebSocket services):

server {
    listen 443 ssl http2;
    # No QUIC, no Alt-Svc
    # WebSocket-specific proxy settings
}

This gives me the performance benefits of HTTP/3 where it works and stability where it doesn't.

Mistakes I Made

I assumed HTTP/3 was a drop-in replacement for HTTP/2. It's not—at least not for WebSocket-heavy applications in 2024.

I also wasted time increasing timeout values when the problem was protocol-level, not timeout-related. The connection wasn't timing out; it was failing silently due to protocol mismatch.

I should have tested WebSocket connections specifically after the upgrade instead of just checking if pages loaded. A simple browser console check would have shown the issue immediately.

Key Takeaways

  • HTTP/3 and WebSockets don't play well together in many applications yet
  • Nginx allows the configuration but doesn't prevent incompatible setups
  • Always test long-lived connections after protocol upgrades
  • Split your Nginx config by use case—not all services need the same protocol support
  • When debugging proxy issues, check protocol negotiation before blaming timeouts

If you're running Home Assistant or Frigate behind Nginx, stick with HTTP/2 for now. The performance gain from HTTP/3 isn't worth the connection stability issues.