Setting up n8n webhooks to trigger home assistant automations when specific systemd services fail or server load exceeds thresholds

I set this to run every 10 minutes. The threshold is 80% of available CPU cores, which works well for my setup. On a 4-core machine, it alerts if load exceeds 3.2.

Processing Alerts in n8n

Back in the n8n workflow, after the Webhook trigger, I added a Switch node to route different alert types:

• Route 1: alert_type equals service_failure
• Route 2: alert_type equals high_load

For service failures, I added a Function node to format the alert message:

const hostname = $input.item.json.hostname;
const service = $input.item.json.service_name;
const status = $input.item.json.status;

return {
  json: {
    message: `Service ${service} on ${hostname} is ${status}`,
    entity_id: "input_boolean.system_alert"
  }
};

For high load alerts, similar logic:

const hostname = $input.item.json.hostname;
const load = $input.item.json.load_avg;
const threshold = $input.item.json.threshold;

return {
  json: {
    message: `High load on ${hostname}: ${load} (threshold: ${threshold})`,
    entity_id: "input_boolean.system_alert"
  }
};

Triggering Home Assistant Automations

I created an input_boolean in Home Assistant's configuration.yaml:

input_boolean:
  system_alert:
    name: System Alert Trigger
    initial: off

In n8n, after the Function node, I added a Home Assistant node configured to call the input_boolean.turn_on service. The node settings:

• Credentials: Home Assistant API token (long-lived access token from HA)
• Resource: services/input_boolean/turn_on
• Entity ID: {{ $json.entity_id }}

I also added a second Home Assistant node to send a persistent notification with the formatted message:

• Resource: services/persistent_notification/create
• Message: {{ $json.message }}
• Title: System Alert

Home Assistant Automation

In Home Assistant, I created an automation that watches for the input_boolean state change:

automation:
  - alias: "System Alert Handler"
    trigger:
      - platform: state
        entity_id: input_boolean.system_alert
        to: "on"
    action:
      - service: notify.mobile_app_my_phone
        data:
          message: "System alert triggered - check Home Assistant"
          title: "Server Alert"
      - service: light.turn_on
        entity_id: light.office_desk_lamp
        data:
          rgb_color: [255, 0, 0]
          brightness: 255
      - delay: "00:00:05"
      - service: input_boolean.turn_off
        entity_id: input_boolean.system_alert

The light turning red is my physical indicator that something needs attention. The 5-second delay before resetting the input_boolean prevents rapid re-triggers if multiple alerts come in quick succession.

What Didn't Work

Polling systemd via n8n directly: I initially tried using n8n's SSH node to poll systemd status on each server every few minutes. This worked but felt clunky—n8n became a bottleneck, and managing SSH keys across multiple nodes was annoying. The push-based webhook approach is cleaner.

Using systemd's OnFailure directive: I experimented with OnFailure= in service unit files to trigger a script on failure. This worked for some services but not others, especially third-party services where I didn't want to modify unit files. The cron-based check is less elegant but more reliable across different service types.

Load average alone isn't enough: I learned that load average can spike temporarily during normal operations (backups, media transcoding). I had to add logic to only alert if load stays high across two consecutive checks. I haven't implemented this yet, but I'm considering a simple state file that tracks previous load values.

Home Assistant webhook vs API: I first tried using Home Assistant's webhook automation trigger directly from the bash scripts. This worked but meant duplicating logic between n8n and Home Assistant. Keeping all the routing and formatting in n8n made it easier to adjust without restarting Home Assistant.

Limitations and Trade-offs

This setup has some clear boundaries:

• No historical data: I'm not storing metrics over time. If I want trends or graphs, I'd need to add something like InfluxDB or Prometheus. Right now, I only care about immediate alerts.
• Single point of failure: If n8n goes down, alerts stop. I could add a fallback that sends email directly from the scripts if the webhook fails, but I haven't done that yet.
• Network dependency: Everything relies on local network connectivity. If my router dies, nothing works. That's acceptable for my use case.
• Cron timing: A 5-minute check interval means I might not know about a failure for up to 5 minutes. For critical services, I could reduce this, but it increases load on n8n.

What I'd Do Differently

If I were starting over, I'd add a simple state management layer to prevent alert spam. Right now, if a service stays failed, I get an alert every 5 minutes until I fix it. A state file that tracks "already alerted" status would help.

I'd also consider using systemd timers instead of cron for the monitoring scripts. Timers give better logging and can handle missed runs more gracefully if a server is down during a scheduled check.

For load monitoring, I'd add a moving average check instead of a single threshold. Two consecutive high readings would be more reliable than one spike.

Key Takeaways

• Push beats poll: Having servers report problems to n8n is simpler than having n8n check every server constantly.
• Keep payloads simple: Flat JSON with only necessary fields made debugging easier.
• Home Assistant as the notification layer works well: It already handles mobile notifications, smart home triggers, and persistent alerts. No need to reinvent that.
• Authentication matters even on local networks: A simple token prevents accidental triggers from other scripts.
• Test failure scenarios: I manually stopped services and artificially increased load to make sure alerts fired correctly. Assumptions about systemd behavior were wrong more than once.

This setup isn't perfect, but it's been running for several months now and has caught real problems—Docker daemon crashes, Nginx misconfigurations, and a VM that was thrashing due to a runaway process. The red light in my office has become a reliable signal that something needs my attention, which was the whole point.