building automated firewall rule testing with docker containers: validating iptables and nftables rulesets before applying to production

Why I Built Automated Firewall Rule Testing

After a late-night production outage caused by a misconfigured iptables rule, I decided enough was enough. Manually testing firewall changes is tedious and error-prone, especially when dealing with complex rulesets across multiple containers. I needed a way to validate rules before they hit production.

My Testing Setup

I created a Docker-based testing environment with:

• A base container running iptables and nftables
• A test container simulating network traffic
• A verification container checking rule effectiveness

Here’s the Dockerfile for my test environment:

FROM alpine:latest
RUN apk add --no-cache iptables nftables curl
COPY ./test-rules.sh /test-rules.sh
CMD ["/test-rules.sh"]

What Worked Well

Rule Validation Framework

I developed a simple validation framework that:

1. Applies test rules to a container
2. Sends test traffic from another container
3. Verifies the rules behave as expected

For example, to test a rule blocking SSH access:

# Apply the rule
iptables -A DOCKER-USER -p tcp --dport 22 -j DROP

# Test from another container
docker exec test-container curl -v http://test-host:22 2>&1 | grep "Connection refused"

# Verify in logs
docker logs test-container | grep "blocked"

Automated Testing Pipeline

I integrated this into my CI/CD pipeline with a GitHub Action:

name: Firewall Rule Test
on: [push]

jobs:
  test-rules:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - run: docker-compose up -d
      - run: docker exec test-container ./run-tests.sh

Challenges and Lessons Learned

Stateful Rule Testing

Testing stateful rules proved tricky. For example, verifying that ESTABLISHED connections work after initial packets are accepted:

# This simple test wasn't enough
iptables -A DOCKER-USER -m state --state ESTABLISHED,RELATED -j ACCEPT

# Needed to simulate full TCP handshake
docker exec test-container tcpreplay --intf1=eth0 handshake.pcap

Performance Impact

Running complex rule tests in containers added overhead. I found that:

• Simple packet tests: <1s per rule
• Full handshake tests: 5-10s per rule
• Concurrent tests: Could overload the host

Key Takeaways

Through this process, I learned:

1. Start with simple packet tests before moving to complex traffic
2. Test both iptables and nftables rulesets separately
3. Simulate real-world traffic patterns for accurate results
4. Integrate testing early in the development cycle

The system isn’t perfect, but it catches 90% of rule misconfigurations before they reach production. The remaining 10% still require manual review, but that’s a vast improvement over where I started.