Firebase cloud functions is very slow

Question

We're working on an application that uses the new firebase cloud functions. What currently is happening is that a transaction is put in the queue node. And then the function removes that node and puts it in the correct node. This has been implemented because of the ability to work offline.

Our current problem is the speed of the function. The function itself takes about 400ms, so that's alright. But sometimes the functions take a very long time (around 8 seconds), while the entry was already added to the queue.

We suspect that the server takes time to boot up, because when we do the action once more after the first. It takes way less time.

Is there any way to fix this problem? Down here i added the code of our function. We suspect there's nothing wrong with it, but we added it just in case.

const functions = require('firebase-functions');
const admin = require('firebase-admin');
const database = admin.database();

exports.insertTransaction = functions.database
    .ref('/userPlacePromotionTransactionsQueue/{userKey}/{placeKey}/{promotionKey}/{transactionKey}')
    .onWrite(event => {
        if (event.data.val() == null) return null;

        // get keys
        const userKey = event.params.userKey;
        const placeKey = event.params.placeKey;
        const promotionKey = event.params.promotionKey;
        const transactionKey = event.params.transactionKey;

        // init update object
        const data = {};

        // get the transaction
        const transaction = event.data.val();

        // transfer transaction
        saveTransaction(data, transaction, userKey, placeKey, promotionKey, transactionKey);
        // remove from queue
        data[`/userPlacePromotionTransactionsQueue/${userKey}/${placeKey}/${promotionKey}/${transactionKey}`] = null;

        // fetch promotion
        database.ref(`promotions/${promotionKey}`).once('value', (snapshot) => {
            // Check if the promotion exists.
            if (!snapshot.exists()) {
                return null;
            }

            const promotion = snapshot.val();

            // fetch the current stamp count
            database.ref(`userPromotionStampCount/${userKey}/${promotionKey}`).once('value', (snapshot) => {
                let currentStampCount = 0;
                if (snapshot.exists()) currentStampCount = parseInt(snapshot.val());

                data[`userPromotionStampCount/${userKey}/${promotionKey}`] = currentStampCount + transaction.amount;

                // determines if there are new full cards
                const currentFullcards = Math.floor(currentStampCount > 0 ? currentStampCount / promotion.stamps : 0);
                const newStamps = currentStampCount + transaction.amount;
                const newFullcards = Math.floor(newStamps / promotion.stamps);

                if (newFullcards > currentFullcards) {
                    for (let i = 0; i < (newFullcards - currentFullcards); i++) {
                        const cardTransaction = {
                            action: "pending",
                            promotion_id: promotionKey,
                            user_id: userKey,
                            amount: 0,
                            type: "stamp",
                            date: transaction.date,
                            is_reversed: false
                        };

                        saveTransaction(data, cardTransaction, userKey, placeKey, promotionKey);

                        const completedPromotion = {
                            promotion_id: promotionKey,
                            user_id: userKey,
                            has_used: false,
                            date: admin.database.ServerValue.TIMESTAMP
                        };

                        const promotionPushKey = database
                            .ref()
                            .child(`userPlaceCompletedPromotions/${userKey}/${placeKey}`)
                            .push()
                            .key;

                        data[`userPlaceCompletedPromotions/${userKey}/${placeKey}/${promotionPushKey}`] = completedPromotion;
                        data[`userCompletedPromotions/${userKey}/${promotionPushKey}`] = completedPromotion;
                    }
                }

                return database.ref().update(data);
            }, (error) => {
                // Log to the console if an error happened.
                console.log('The read failed: ' + error.code);
                return null;
            });

        }, (error) => {
            // Log to the console if an error happened.
            console.log('The read failed: ' + error.code);
            return null;
        });
    });

function saveTransaction(data, transaction, userKey, placeKey, promotionKey, transactionKey) {
    if (!transactionKey) {
        transactionKey = database.ref('transactions').push().key;
    }

    data[`transactions/${transactionKey}`] = transaction;
    data[`placeTransactions/${placeKey}/${transactionKey}`] = transaction;
    data[`userPlacePromotionTransactions/${userKey}/${placeKey}/${promotionKey}/${transactionKey}`] = transaction;
}

Answer 1

firebaser here

It sounds like you're experiencing a so-called cold start of the function.

When your function hasn't been executed in some time, Cloud Functions puts it in a mode that uses fewer resources so that you don't pay for compute time that you're not using. Then when you hit the function again, it restores the environment from this mode. The time it takes to restore consists of a fixed cost (e.g. restore the container) and a part variable cost (e.g. if you use a lot of node modules, it may take longer).

We're continually monitoring the performance of these operations to ensure the best mix between developer experience and resource usage. So expect these times to improve over time.

The good news is that you should only experience this during development. Once your functions are being frequently triggered in production, chances are they'll hardly ever hit a cold start again, especially if they have consistent traffic. If some functions tend to see spikes of traffic, however, you'll still see cold starts for every spike. In that case, you may want to consider the minInstances setting to keep a set number of instances of a latency-critical function warm at all times.

Answer 2

Update March 2021 It may be worth checking out the answer below from @George43g which offers a neat solution to automating the below process. Note - I haven't tried this myself and so cannot vouch for it, but it seems to automate the process described here. You can read more at https://github.com/gramstr/better-firebase-functions - otherwise read on for how to implement it yourself and understand what is happening inside functions.

Update May 2020 Thanks for the comment by maganap - in Node 10+ FUNCTION_NAME is replaced with K_SERVICE (FUNCTION_TARGET is the function itself, not it's name, replacing ENTRY_POINT). Code samples below have been udpated below.

More info at https://cloud.google.com/functions/docs/migrating/nodejs-runtimes#nodejs-10-changes

Update - looks like a lot of these problems can be solved using the hidden variable process.env.FUNCTION_NAME as seen here: https://github.com/firebase/functions-samples/issues/170#issuecomment-323375462

Update with code - For example, if you have the following index file:

...
exports.doSomeThing = require('./doSomeThing');
exports.doSomeThingElse = require('./doSomeThingElse');
exports.doOtherStuff = require('./doOtherStuff');
// and more.......

Then all of your files will be loaded, and all of those files' requirements will also be loaded, resulting in a lot of overhead and polluting your global scope for all of your functions.

Instead separating your includes out as:

const function_name = process.env.FUNCTION_NAME || process.env.K_SERVICE;
if (!function_name || function_name === 'doSomeThing') {
  exports.doSomeThing = require('./doSomeThing');
}
if (!function_name || function_name === 'doSomeThingElse') {
  exports.doSomeThingElse = require('./doSomeThingElse');
}
if (!function_name || function_name === 'doOtherStuff') {
  exports.doOtherStuff = require('./doOtherStuff');
}

This will only load the required file(s) when that function is specifically called; allowing you to keep your global scope much cleaner which should result in faster cold-boots.

---

This should allow for a much neater solution than what I've done below (though the explanation below still holds).

---

Original Answer

It looks like requiring files and general initialisation happening in the global scope is a huge cause of slow-down during cold-boot.

As a project gets more functions the global scope is polluted more and more making the problem worse - especially if you scope your functions into separate files (such as by using Object.assign(exports, require('./more-functions.js')); in your index.js.

I've managed to see huge gains in cold-boot performance by moving all my requires into an init method as below and then calling it as the first line inside any function definition for that file. Eg:

const functions = require('firebase-functions');
const admin = require('firebase-admin');
// Late initialisers for performance
let initialised = false;
let handlebars;
let fs;
let path;
let encrypt;

function init() {
  if (initialised) { return; }

  handlebars = require('handlebars');
  fs = require('fs');
  path = require('path');
  ({ encrypt } = require('../common'));
  // Maybe do some handlebars compilation here too

  initialised = true;
}

I've seen improvements from about 7-8s down to 2-3s when applying this technique to a project with ~30 functions across 8 files. This also seems to cause functions to need to be cold-booted less often (presumably due to lower memory usage?)

Unfortunately this still makes HTTP functions barely usable for user-facing production use.

Hoping the Firebase team have some plans in future to allow for proper scoping of functions so that only the relevant modules ever need to be loaded for each function.

Answer 3

I am facing similar issues with firestore cloud functions. The biggest is performance. Specially in case of early stage startups, when you can't afford your early customers to see "sluggish" apps. A simple documentation generation function for e.g gives this:

-- Function execution took 9522 ms, finished with status code: 200

Then: I had a straighforward terms and conditions page. With cloud functions the execution due to the cold start would take 10-15 seconds even at times. I then moved it to a node.js app, hosted on appengine container. The time has come down to 2-3 seconds.

I have been comparing many of the features of mongodb with firestore and sometimes I too wonder if during this early phase of my product I should also move to a different database. The biggest adv I had in firestore was the trigger functionality onCreate, onUpdate of document objects.

https://db-engines.com/en/system/Google+Cloud+Firestore%3BMongoDB

Basically if there are static portions of your site that can be offloaded to appengine environment, perhaps not a bad idea.

Answer 4

UPDATE: 2022 - lib is maintained again. Firebase now has the ability to keep instances warm, however there's still potential performance & code structure benefits.

UPDATE/EDIT: new syntax and updates coming MAY2020

I just published a package called better-firebase-functions, it automatically searches your function directory and correctly nests all the found functions in your exports object, while isolating the functions from each other to improve cold-boot performance.

If you lazy-load and cache only the dependencies you need for each function within the module scope, you'll find it's the simplest and easiest way to keep your functions optimally efficient over a fast-growing project.

import { exportFunctions } from 'better-firebase-functions'
exportFunctions({__filename, exports})

Answer 5

I have done these things as well, which improves performance once the functions are warmed up, but the cold start is killing me. One of the other issues I've encountered is with cors, because it takes two trips to the cloud functions to get the job done. I'm sure I can fix that, though.

When you have an app in its early (demo) phase when it is not used frequently, the performance is not going to be great. This is something that should be considered, as early adopters with early product need to look their best in front of potential customers/investors. We loved the technology so we migrated from older tried-and-true frameworks, but our app seems pretty sluggish at this point. I'm going to next try some warm-up strategies to make it look better

Answer 6

I experienced a very poor performance for my first project in Firebase Functions, where a simple function would be executed in minutes (knowing the 60s limit for function execution, I knew there was something wrong with my functions). The issue for my case is I didn't properly terminate the function

In case someone experienced the same issue, make sure to terminate the function by either:

1. Sending a response for HTTP triggers
2. Returning a promise for background triggers

Here's the youtube link from Firebase that helped me solve the issue

Answer 7

Cloud Functions have inconsistent cold start times when used with firestore libraries due to the gRpc libraries used within it.

We recently made a full compatible Rest client (@bountyrush/firestore) which is aimed to get updated in parallel to official nodejs-firestore client.

Fortunately, the cold starts are much better now and we even dropped using redis memory store as cache which we used earlier.

Steps to integrate:

1. npm install @bountyrush/firestore
2. Replace require('@google-cloud/firestore') with require('@bountyrush/firestore')
3. Have FIRESTORE_USE_REST_API = 'true' in your environment variables. (process.env.FIRESTORE_USE_REST_API should be set to 'true' for using in rest mode. If its not set, it just standard firestore with grpc connections)