Guide: Building a High-Availability Nginx Proxy Manager (NPM) on Proxmox

As the “Cheerio Lab” grows, the single-point-of-failure (SPOF) of a Docker-based Nginx Proxy Manager becomes a bottleneck. Transitioning to a High-Availability (HA) LXC setup reduces overhead and ensures that if a Proxmox node reboots, the external traffic (like the Stable Diffusion batch runner or Gitea) stays online.

1. The Core Architecture: Shared Virtual IP (VIP)

The secret to HA without reconfiguring the router is Keepalived. It allows two separate LXCs to share a single “Floating” IP address.

  • Primary LXC (.10): Handles traffic normally.
  • Backup LXC (.11): Monitors the Primary via Heartbeat.
  • The Virtual IP (.12): This is the IP you point your router Port Forwarding (80/443) toward.

If the Primary stops responding, the Backup instantly claims .12. The router doesn’t care; it just keeps sending traffic to the same IP.

2. Solving the Storage Dilemma

Previously considered using NFS, but for networking, it’s a trap. A Loop Dependency occurs if your NPM needs the NAS to start, but the NAS relies on NPM for routing. Additionally, NFS latency makes the UI feel sluggish.

The Solution: Local Storage + Rsync Instead of a shared drive, we keep /data and /letsencrypt folders on the local SSD of each Proxmox node for maximum speed. We use an rsync cron job to push changes from the Primary to the Backup every few minutes.

3. Implementation Steps

A. Setup the LXCs

Deploy two identical Ubuntu LXCs using a lightweight template (approx. 256MB RAM is plenty). Install NPM on both.

B. Configure Keepalived

Install it on both: sudo apt install keepalived.

Primary Config (/etc/keepalived/keepalived.conf):

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    virtual_ipaddress {
        192.168.X.12/24  # Your Floating VIP
    }
}

The Secondary (Backup) keepalived.conf: On the second LXC (e.g., 192.168.X.11), create the file at /etc/keepalived/keepalived.conf:

vrrp_instance VI_1 {
    state BACKUP          # Set to BACKUP
    interface eth0        # Ensure this matches your network interface name
    virtual_router_id 51  # MUST match the Primary exactly, and is unique for every HA service
    priority 50           # Lower than Master (100) so it yields to Primary
    advert_int 1          # Check in with master every 1 second (all nodes within the same vrrp instance needs the same advert_int)
    virtual_ipaddress {
        192.168.X.12/24   # The same Floating VIP
    }
}

C. The Sync Script

On the Primary, create a script to keep the Backup updated:

rsync -avz --delete /opt/npm/data/ root@192.168.X.11:/opt/npm/data/
rsync -avz --delete /opt/npm/letsencrypt/ root@192.168.X.11:/opt/npm/letsencrypt/

Notes:

  • Ensure SSH keys are exchanged between LXCs so this runs without a password.
  • For the NPM transition to be truly successful, remember that NPM uses an SQLite database by default (stored in /data/database.sqlite) and needs to be restarted post-sync.
  • Add systemctl reload openresty (the engine behind NPM) to the end of the rsync script on the Backup node to make the transition seamless

How it Works (The “Future Me” Logic)

  • The Priority Tug-of-War: Keepalived uses the priority value to decide who “owns” the VIP. Since the Primary is at 100 and the Backup is at 50, the Primary will always take the IP if it is healthy.
  • The Heartbeat: The nodes “talk” to each other via a multicast signal every second (advert_int 1).
  • The Failover: If the Backup stops hearing the Master for 3 seconds, it assumes the Master is dead and instantly promotes itself to MASTER, grabbing 192.168.X.12.

4. Why Future Me Is Doing This

  • Lower Overhead: Transitioning from a VM/Docker environment to LXC drops RAM usage by ~80%.
  • Resilience: Proxmox updates no longer take down the whole lab.
  • Simplicity: Router settings never change because they only see the VIP.

Example /etc/keepalived/keepalived.conf for multiple HA services on the same primary node

//Service 1: NPM 
vrrp_instance VI_NPM {
    state MASTER
    interface eth0
    virtual_router_id 60  # Unique ID
    priority 100
    advert_int 1
    virtual_ipaddress {
        192.168.X.12/24
    }
}

// Service 2: A different service or VLAN
vrrp_instance VI_SERVICE {
    state MASTER
    interface eth0
    virtual_router_id 70  # Different Unique ID
    priority 100
    advert_int 1
    virtual_ipaddress {
        192.168.X.20/24
    }
}

For securing the HA service (since any node on the network could technically send a ‘higher priority’ signal to hijack the IP. A password block can be added to ensure only authorized LXCs can participate in the election.

vrrp_instance VI_NPM {
    state MASTER
    interface eth0
    virtual_router_id 60
    priority 100
    advert_int 1
    
    // The Password Block
    authentication {
        auth_type PASS
        auth_pass MySecure123  # Limit this to 8 characters for compatibility
    }

    virtual_ipaddress {
        192.168.X.12/24
    }
}

Best Practices:

  • 8-Character Limit: While some versions of Keepalived support longer strings, the standard VRRP protocol technically only uses the first 8 characters of the password. Stick to a 6–8 character alphanumeric string to avoid weird “mismatch” errors where the Backup refuses to acknowledge the Master.
  • Matches are Mandatory: The auth_type and auth_pass must be identical on both the Master and the Backup. If they don’t match, both nodes will think they are the Master, creating an IP Conflict (Split-Brain) on your network.
  • Unique Passwords per Service: If you are running multiple instances (e.g., VI_NPM and VI_SERVICE), use a different password for each. This prevents a misconfigured config file in one service from accidentally interfering with another.

5. Service Centric

  • For specific service health checks and not just the node, utilize track script to verify the service is alive.
vrrp_script check_npm {
    script "pidof openresty" # NPM runs on openresty
    interval 2               # Check every 2 seconds
    weight -20               # If it fails, drop priority by 20
}

vrrp_instance VI_1 {
    ...
    track_script {
        check_npm
    }
}

Failures trigger now in 3 instances:

  • The node hangs, or hosts reboot, keepalived stops sending hearbeats and the backup takesover in ~3 seconds.
  • The service crashes, (node is running), the track script fails and nodes priority drops, triggering failover.
  • The network fails, if the network connection between the two nodes fails, the Backup assumes Master is gone. This could trigger split-mode. However, if you’re network is down, the notion of master/backup service doesn’t mean much if there’s not service being served.