Email Queues and Async Sending in Node.js Bull, SQS, and Background Jobs

Email Queues and Async Sending in Node.js

Sending emails synchronously in route handlers blocks the response. Email queues solve this by processing emails in the background.

Why Use Email Queues?

1. Faster responses Don't block HTTP responses on email sending
2. Retry on failure Automatically retry failed emails
3. Rate limiting Control how many emails are sent per second
4. Priority Send important emails first
5. Scalability Separate workers for email processing
6. Persistence Emails survive server restarts

Option 1: Fire and Forget (Simplest)

// Simplest approach   don't await, just catch errors
router.post("/signup", asyncHandler(async (req, res) => {
  const user = await User.create(req.body);

  // Fire and forget
  sendEmail(user.email, "welcome", { firstName: user.firstName })
    .catch(err => console.error("Email failed:", err));

  res.status(201).json({ data: user });
}));

Pros: Simple, no extra dependencies Cons: No retry, lost on server restart, no rate limiting

Option 2: In-Memory Queue (Simple)

// src/utils/emailQueue.js
const emailQueue = [];
let processing = false;

const processQueue = async () => {
  if (processing || emailQueue.length === 0) return;
  processing = true;

  while (emailQueue.length > 0) {
    const email = emailQueue.shift();
    try {
      await sendEmail(email.to, email.template, email.data);
    } catch (err) {
      console.error("Email failed:", err);
      // Could add retry logic here
    }
  }

  processing = false;
};

const queueEmail = (to, template, data) => {
  emailQueue.push({ to, template, data });
  processQueue(); // Start processing if not already
};

module.exports = { queueEmail };

Pros: Simple, no external dependencies Cons: Lost on restart, no persistence, no retry

Option 3: Bull Queue with Redis (Recommended)

npm install bull

// src/queues/emailQueue.js
const Queue = require("bull");
const { sendEmail } = require("../utils/emailTemplates");

const emailQueue = new Queue("email", {
  redis: { host: "127.0.0.1", port: 6379 }
});

// Process jobs
emailQueue.process(async (job) => {
  const { to, template, data } = job.data;
  await sendEmail(to, template, data);
  return { sent: true };
});

// Handle failures
emailQueue.on("failed", (job, err) => {
  console.error(`Email job ${job.id} failed: ${err.message}`);
});

// Add jobs
const queueEmail = (to, template, data) => {
  return emailQueue.add({ to, template, data }, {
    attempts: 3,           // Retry 3 times
    backoff: {
      type: "exponential",
      delay: 5000          // 5s, 10s, 20s
    },
    removeOnComplete: true
  });
};

module.exports = { queueEmail };

Usage:

const { queueEmail } = require("../queues/emailQueue");

router.post("/signup", asyncHandler(async (req, res) => {
  const user = await User.create(req.body);
  
  // Queue email instead of sending directly
  await queueEmail(user.email, "welcome", { firstName: user.firstName });
  
  res.status(201).json({ data: user });
}));

Pros: Persistent (Redis), automatic retry, rate limiting, priority, scalable Cons: Requires Redis

Option 4: AWS SQS (AWS-native)

npm install @aws-sdk/client-sqs

const { SQSClient, SendMessageCommand } = require("@aws-sdk/client-sqs");

const sqs = new SQSClient({ region: "us-east-1" });

const queueEmail = async (to, template, data) => {
  const params = {
    QueueUrl: process.env.EMAIL_QUEUE_URL,
    MessageBody: JSON.stringify({ to, template, data })
  };
  await sqs.send(new SendMessageCommand(params));
};

// Separate worker process
const { ReceiveMessageCommand, DeleteMessageCommand } = require("@aws-sdk/client-sqs");

const processEmails = async () => {
  const result = await sqs.send(new ReceiveMessageCommand({
    QueueUrl: process.env.EMAIL_QUEUE_URL,
    MaxNumberOfMessages: 10,
    WaitTimeSeconds: 20
  }));

  for (const message of result.Messages || []) {
    const { to, template, data } = JSON.parse(message.Body);
    try {
      await sendEmail(to, template, data);
      await sqs.send(new DeleteMessageCommand({
        QueueUrl: process.env.EMAIL_QUEUE_URL,
        ReceiptHandle: message.ReceiptHandle
      }));
    } catch (err) {
      console.error("Email failed:", err);
      // Message returns to queue after visibility timeout
    }
  }

  processEmails(); // Continue polling
};

Pros: Fully managed, no Redis needed, integrates with AWS Cons: More complex setup, polling latency

Which Queue to Choose?

The Takeaway

For most Node.js apps, Bull with Redis is the recommended email queue. It provides persistence (survives restarts), automatic retry with exponential backoff, rate limiting, and priority. For small apps, fire-and-forget is sufficient. For AWS-heavy apps, SQS integrates seamlessly with SES and other AWS services.