Delete old data with Firebase Cloud Functions

Question

I need to delete old data from Firebase regularly. I found this solution but have not been able to make it work.

Terminal tells me functions[deleteOldItems(us-central1)]: Successful update operation.But the data is not deleted.

This is what I have done:

const functions = require('firebase-functions');

exports.deleteOldItems = functions.database.ref('/nodeThatContainsDataToBeDeleted')
.onWrite((change, context) => {
  var ref = change.after.ref.parent; // reference to the items
  var now = Date.now();
  var cutoff = now - 2 * 60 * 60 * 1000;
  var oldItemsQuery = ref.orderByChild('timestamp').endAt(cutoff);
  return oldItemsQuery.once('value', function(snapshot) {
    // create a map with all children that need to be removed
    var updates = {};
    snapshot.forEach(function(child) {
      updates[child.key] = null
    });
    // execute all updates in one go and return the result to end the function
    return ref.update(updates);
  });
});

The timestamp is 639915248.124176 (from Swift)

Database structure:

root : {
         "nodeThatContainsDataToBeDeleted" : {
            "-r5tyfX9FGC0glhgf78Ia" : {
              "company" : "CompanyCo",
              "name" : "Sam",
              "phone" : "1212",
              "imageURL" : "https://firebasestorage.googleapis.com/imageurl",
              "timestamp" : 6.39915248124176E8
            }
          },
    }

Answer 1

When you execute asynchronous operations in a background triggered Cloud Function (in your case the once() and update() methods), you must return a Promise, in such a way the Cloud Function waits that this promise resolves in order to terminate. More info in the three official videos titled "Learn JavaScript Promises" (Parts 2 & 3 especially focus on background triggered Cloud Functions, but it really worth watching Part 1 before).

By doing oldItemsQuery.once('value', function(snapshot) {...}) you are actually using the callback version of the once() method.

In order to correctly chain the different promises and return this chain in the callback, you need to use the promise version, as follows:

exports.deleteOldItems = functions.database.ref('/nodeThatContainsDataToBeDeleted')
    .onWrite((change, context) => {
        var ref = change.after.ref.parent; // reference to the items
        var now = Date.now();
        var cutoff = now - 2 * 60 * 60 * 1000;
        var oldItemsQuery = ref.orderByChild('timestamp').endAt(cutoff);
        return oldItemsQuery.once('value')
            .then(snapshot => {
                // create a map with all children that need to be removed
                var updates = {};
                snapshot.forEach(function (child) {
                    updates[child.key] = null
                });
                // execute all updates in one go and return the result to end the function
                return ref.update(updates);
            });
    });

Answer 2

This answer builds upon @RenaudTarnec's answer.

JavaScript and the Firebase SDKs have come a long way since 2015 so be wary of this when copying/making use of old code examples on StackOverflow. The solution you linked also links to the functions-samples repo where that example code has been kept up to date with modern features.

As Renaud covered, you need to switch to exclusively using the Promise version of the once() listener instead of using a callback. While your code does return a Promise as expected, the current callback isn't inserted into the Promise chain correctly, which means your Cloud Function doesn't wait for the code in the callback to finish. The linked example was written in 2015 where this used to work, but that is no longer the case.

These lines:

return oldItemsQuery.once('value', function(snapshot) {
    /* ... use snapshot ... */
  });

should be:

return oldItemsQuery.once('value')
  .then((snapshot) => {
    /* ... use snapshot ... */
  });

or:

const snapshot = await oldItemsQuery.once('value');
/* ... use snapshot ... */

Next, you have configured your onWrite listener to listen to changes on /nodeThatContainsDataToBeDeleted. While this works, it's terribly inefficient. For every small change in your database at some point under /nodeThatContainsDataToBeDeleted (such as /nodeThatContainsDataToBeDeleted/somePushId/someBoolean = true), your function downloads all the data nested under that node. With a few entries at this node, this is insignificant, but as you approach thousands of entries, you can be reading many megabytes of data.

If you look at the example you linked, the listener is attached to /path/to/items/{pushId} (the single written entry) instead of /path/to/items (all of the entries). The {pushId} past is a named wildcard that captures the last part of the path (like -r5tyfX9FGC0glhgf78Ia).

This line:

functions.database.ref('/nodeThatContainsDataToBeDeleted') // fetches all entries

should be:

functions.database.ref('/nodeThatContainsDataToBeDeleted/{pushId}') // fetches only the entry that changed

Note: One "bug" with this function is that it retriggers itself. For each entry it deletes, deleteOldItems will be fired again. You may wish to use either .onCreate() or .onUpdate() instead of .onWrite() - see the docs for more info.

Combining these changes gives (built on the latest sample code at the time of writing):

// Copyright 2017 Google Inc., Apache 2.0
// Sourced at https://github.com/firebase/functions-samples/blob/master/delete-old-child-nodes/functions/index.js

// Cut off time. Child nodes older than this will be deleted.
const CUT_OFF_TIME = 2 * 60 * 60 * 1000; // 2 Hours in milliseconds.

/**
 * This database triggered function will check for child nodes that are older than the
 * cut-off time. Each child needs to have a `timestamp` attribute.
 */
exports.deleteOldItems = functions.database.ref('/nodeThatContainsDataToBeDeleted/{pushId}').onWrite(async (change) => {
  const ref = change.after.ref.parent; // reference to the parent
  const now = Date.now();
  const cutoff = now - CUT_OFF_TIME;
  const oldItemsQuery = ref.orderByChild('timestamp').endAt(cutoff);
  const snapshot = await oldItemsQuery.once('value');
  // create a map with all children that need to be removed
  const updates = {};
  snapshot.forEach(child => {
    updates[child.key] = null;
  });
  // execute all updates in one go and return the result to end the function
  return ref.update(updates);
});