Nginx Load Balancing Running Multiple Node.js Instances for High Availability

Nginx Load Balancing for Multiple Node.js Instances

Node.js is single-threaded, meaning it uses only one CPU core. To use all cores on your server, run multiple Node.js instances and use Nginx to load balance between them.

Why Load Balance Node.js?

1. Use all CPU cores A t3.medium has 2 vCPUs. One Node.js instance uses 1. Run 2 instances to use both.
2. High availability If one instance crashes, others continue serving
3. Zero-downtime deployment Update instances one at a time
4. Better throughput More requests processed in parallel

Using PM2 Cluster Mode

The easiest way to run multiple instances:

pm2 start src/server.js --name devtinder-api -i max

-i max starts one instance per CPU core. PM2 handles load balancing internally using the cluster module.

Alternatively, specify the number:

pm2 start src/server.js --name devtinder-api -i 2

Using Nginx for Load Balancing

For more control, run instances on different ports and use Nginx:

# Start multiple instances on different ports
PORT=3000 pm2 start src/server.js --name api-1
PORT=3001 pm2 start src/server.js --name api-2
PORT=3002 pm2 start src/server.js --name api-3

Nginx Load Balancing Configuration

upstream devtinder_backend {
    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
    server 127.0.0.1:3002;
}

server {
    listen 80;
    server_name api.yourdomain.com;

    location / {
        proxy_pass http://devtinder_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Load Balancing Methods

1. Round-Robin (Default)

Distributes requests in order: 1, 2, 3, 1, 2, 3, ...

upstream backend {
    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
    server 127.0.0.1:3002;
}

2. Least Connections

Sends to the server with the fewest active connections:

upstream backend {
    least_conn;
    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
    server 127.0.0.1:3002;
}

3. IP Hash

Same client always goes to the same server (useful for sessions):

upstream backend {
    ip_hash;
    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
    server 127.0.0.1:3002;
}

4. Weighted

Distribute based on server capacity:

upstream backend {
    server 127.0.0.1:3000 weight=3;  # 3x more requests
    server 127.0.0.1:3001 weight=1;
    server 127.0.0.1:3002 weight=1;
}

Health Checks

Nginx automatically removes servers that fail. Configure health checks:

upstream backend {
    server 127.0.0.1:3000 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3001 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3002 max_fails=3 fail_timeout=30s;
}

max_fails=3 Remove server after 3 failed attempts fail_timeout=30s Time window for fail count and recovery time

Keep-Alive to Backend

upstream backend {
    server 127.0.0.1:3000;
    keepalive 32;
}

server {
    location / {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

This maintains 32 persistent connections to the backend, reducing connection overhead.

WebSocket Load Balancing

For Socket.io, use ip_hash to ensure the same client always connects to the same instance:

upstream backend {
    ip_hash;
    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
}

server {
    location /socket.io/ {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Zero-Downtime Deployment

# Update one instance at a time
pm2 reload api-1
# Wait for it to be ready
pm2 reload api-2
pm2 reload api-3

Or with Nginx, remove servers one at a time:

# Comment out one server, reload Nginx
upstream backend {
    # server 127.0.0.1:3000;  # Temporarily removed
    server 127.0.0.1:3001;
    server 127.0.0.1:3002;
}

The Takeaway

Nginx load balancing lets you use all CPU cores by running multiple Node.js instances. Use PM2 cluster mode (-i max) for simplicity, or Nginx upstream blocks for more control. Choose round-robin (default), least_conn, ip_hash, or weighted based on your needs. Configure health checks with max_fails and fail_timeout, and use ip_hash for WebSocket connections.