Dynamic Scheduled Background Jobs in Firebase

Last week, Firebase announced a new scheduled cron trigger for Cloud Functions that makes it easy to run serverless code on a set time interval. This function type is special because it combines the powers of Cloud Scheduler and Pub/Sub to guarantee security that you don’t have with a regular HTTP-triggered function.

Scheduling a function on a static time interval is straight forward, but what if you want to build a dynamic task queue where users can schedule their own background jobs? For example, you might want to…

• allow users to customize times for transactional email delivery
• schedule push notifications or similar alerts dynamically
• enqueue background jobs to run at specific times
• build robocallers 🤣 - please don’t

Basic Scheduled Function

Let’s start by looking at an example of a basic cron-scheduled Cloud Function.

Make sure you have the latest version of firebase-tools (or at least version 6.7) installed on your system, then initialize a new project.

Learn more about cron schedules.

npm i firebase-tools@latest -g

firebase init functions

A basic scheduled Cloud Function can be defined on the pubsub namespace.

export const dailyJob = functions.pubsub

   .schedule('30 5 * * *').onRun(context => {
      console.log('This will be run every day at 5:30AM');
   });

export const everyFiveMinuteJob = functions.pubsub

    .schedule('every 5 minutes').onRun(context => {
        console.log('This will be run every 5 minutes!');  
     });

Dynamic Task Queue

Our task queue or job queue is simply a Firestore collection that will be queried by a Pub/Sub Cloud Function every 60 seconds. If the current time is greater than the performAt time of a task, then we execute it.

Step 1: Data Model for Background Jobs

A task is a generic document that tells the Cloud Function how to run the backgorund code.

• performAt when to execute the task as a Firestore timestamp.
• status the state of the tasks, useful for debugging and/or querying.
• worker the name of worker function, which contains the business logic to execute. See step 3.
• options a map containing extra data for the worker function, like a userID argument for example.

tasks/{taskID}/

   performAt: TimeStamp
   status: 'scheduled' | 'complete' | 'error'
   worker: string 
   options: Map

Step 2: Task Runner Cloud Function

Next, we need to define a Pub/Sub Cloud Function that queries the task collection every 60s (or whatever granularity you want) for tasks that are ready to perform.

import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
admin.initializeApp();
const db = admin.firestore();

export const taskRunner = functions.runWith( { memory: '2GB' }).pubsub

    .schedule('* * * * *').onRun(async context => {

        // Consistent timestamp
        const now = admin.firestore.Timestamp.now();
        
        // Query all documents ready to perform
        const query = db.collection('tasks').where('performAt', '<=', now).where('status', '==', 'scheduled');

        const tasks = await query.get();


        // Jobs to execute concurrently. 
        const jobs: Promise<any>[] = [];

        // Loop over documents and push job.
        tasks.forEach(snapshot => {
            const { worker, options } = snapshot.data();

            const job = workers[worker](options)
                
                // Update doc with status on success or error
                .then(() => snapshot.ref.update({ status: 'complete' }))
                .catch((err) => snapshot.ref.update({ status: 'error' }));

            jobs.push(job);
        });

        // Execute all jobs concurrently
        return await Promise.all(jobs);

});

Keep in mind, this is a compound query that will require a Firestore index.

Step 3: Define Worker Functions to Run Jobs

Now that we have a working function in place, we can define the business logic (worker functions) that execute a task.

// Optional interface, all worker functions should return Promise. 
interface Workers {
    [key: string]: (options: any) => Promise<any>
}

// Business logic for named tasks. Function name should match worker field on task document. 
const workers: Workers = {
    helloWorld: () => db.collection('logs').add({ hello: 'world' }),
}

Run firebase deploy --only functions.

After the function is deployed we just need to create a task document in Firestore that points the helloWorld worker. Within 1 minute you should see the task document update to complete