How to Implement Scheduled Jobs in Node.js

Many applications need tasks that run automatically on a schedule rather than in response to a user request. Common examples include sending daily email digests, generating weekly reports, cleaning up expired database records, syncing data from external APIs, sending reminder notifications, and rotating log files. These tasks run in the background independent of the main HTTP server.

Install the node-cron package for scheduling tasks using cron syntax. Call cron.schedule with a cron expression and a callback function. The cron expression defines when the task runs using five fields representing minute, hour, day of month, month, and day of week. An asterisk means every unit of that field. For example, the expression '0 9 * * 1' runs at 9 AM every Monday.

A cron expression has five space-separated fields. The first field is minutes from 0 to 59. The second is hours from 0 to 23. The third is the day of the month from 1 to 31. The fourth is the month from 1 to 12. The fifth is the day of the week from 0 to 7 where both 0 and 7 represent Sunday. Use an asterisk for any field to mean every possible value for that unit.

If a scheduled task takes longer to complete than the interval between runs, the next execution starts before the previous one finishes. Implement a lock mechanism to prevent overlapping executions. Set a flag in Redis at the start of the task and release it when done. If the flag is already set when the next execution starts, skip that execution and log a warning.

node-cron is simple but jobs are lost if the server restarts. For reliable job processing, use Bull which is backed by Redis. Install bull and create a Queue instance with a name and Redis configuration. Add jobs to the queue using queue.add. Define a processor function using queue.process that receives each job and performs the work. Bull persists jobs in Redis so they survive server restarts and can be retried on failure.

When adding a job to a Bull queue, pass an options object as the third argument. To delay a job, set the delay property to the number of milliseconds to wait. To create a repeating job similar to cron, set the repeat property with a cron string or an every property with milliseconds. Bull manages the scheduling in Redis so recurring jobs work correctly even when your application restarts.

Add an attempts property to the job options to specify how many times Bull should retry a failed job. Add a backoff property to define the delay strategy between retries. Use type 'exponential' with a delay value to implement exponential backoff, where each retry waits twice as long as the previous one. This prevents hammering a temporarily unavailable external service with immediate retries.

Use Bull's event system to log job lifecycle events. Listen for the completed event to log successful completions with the job ID and duration. Listen for the failed event to log failures with the error message and the number of attempts made. Use the Bull Board package to add a visual dashboard to your Express application showing the status of all queues and individual jobs, making it easy to diagnose stuck or failing jobs.