Handle Deadline exceeded Firebase Cloud Functions

Question

In using firebase functions to insert data in my firebase firestore. They take data from differents APIs. I write and read a lot of documents - probably why I get this error. I've read a lot of Stackoverflow posts and issues on Github too and I found that this error could have many reasons. I'm 99% sure that this as nothing to do with Promises. I'm using async / await

How can I handle it, what are the solutions?

The exact errors when I catch it : Error: 4 DEADLINE_EXCEEDED: Deadline exceeded Firebase Cloud Functions

Here is the code l1b3rty asked :


if (plans.results.length != 0) {
    const batch = db.batch()
    plans.results.forEach(async (plan) => {
        const jiraKey = plan.planItem.self.split('/').pop()
        let wyzioProjectId
        const projects = await db.collection('projects').where('jiraKey', '==', jiraKey).get()
        if (!projects.empty) {
            projects.forEach(async (project) => {
            wyzioProjectId = project.data().wyzioId
            const jiraUserId = plan.assignee.self.split('=').pop()
            const username = await user.findOne('user?accountId=' + jiraUserId)
            if (username != null) {
                let wyzioUserId
                const users = await db.collection('users').where('name', '==', username).get()
                if (!users.empty) {
                    users.forEach(user => {
                        wyzioUserId = user.data().id
                    })
                }
       batch.set(db.collection('plans').doc(JSON.stringify(plan.id)), {
                        'TempoId': plan.id,
                        'jiraKey': jiraKey,
                        'projectId': wyzioProjectId,
                        'userId': wyzioUserId,
                        'username': username,
                        'startDate': plan.startDate,
                        'values': plan.dates.values,
                        'status': 1,
                        'syncStatus': 1
                    }, { merge: true })
                }
            })
         } else {
            let key = "tempo_plan_" + plan.jiraWorkLogId + "_no_ref_project"
            let message = "Impossible d'ajouter le plan : " + plan.jiraWorkLogId + ", il n'appartient à aucun projet de la base de données"
            await notifications.addNotifications(db, key, message)
        }
    })
    try {
        await batch.commit()
        await incrementOffsetTempo(db, 'tempo_plans')
        console.log("--- tempo_plans_done ---")
    } catch {
        let key = "tempo_plan_" + plan.jiraWorkLogId + "_error_sync"
        let message = "Impossible d'ajouter le plan : " + plan.jiraWorkLogId + ", veuillez contrôler ses informations"
        await notifications.addNotifications(db, key, message)
    }
}

The batch.set() is not correctly aligned

I'm thinking that Firebase Cloud Functions are actually GCP Cloud Functions behind the scenes. If that is true, then maybe we could login to your GCP project and then look at the Cloud Logging logs and see what Cloud Logging might be reporting. This might give us more data to work upon. — Kolban, Oct 15 '21 at 15:12
I think I found the GCP logs and there are 2 different errors : 1. `Error: socket hang up at connResetException` 2. `Error: write EPROTO at WriteWrap.onWriteComplete [as completed]` — MVT KVM, Oct 15 '21 at 15:17
Have you tried increasing the timeout and memory allocation of hte function? You can do that directly in the code or via the GCP console — l1b3rty, Oct 15 '21 at 15:43
And do your read/writes have to happen in sequence? Because if they can happen in parallel, it would go much faster to use Promises instead of async / await — l1b3rty, Oct 15 '21 at 15:44
It seems a lot of reads and writes are causing the error. Please refer to this [StackOverFlow thread](https://stackoverflow.com/questions/67716856/firebase-cloud-functions-4-deadline-exceeded-deadline-exceeded). — Prabir, Oct 16 '21 at 16:52
@Prabir I've seen this thread, however, they don't really give a solution. What are the thing I can do to solve this? — MVT KVM, Oct 18 '21 at 06:02
@l1b3rty I have both, some require other datas from the firestore and some doesn't. With my error handling (adding logs in a collection) I've seen that many errors happen when the read / writes are in sequence. Also I did increased the timeout, I'll check for the memory now — MVT KVM, Oct 18 '21 at 06:05
Can you share the code of the Cloud Function triggering the error? — l1b3rty, Oct 18 '21 at 13:12
@l1b3rty I have now resolved this problem using batched write when I could. Now I have an error `Cannot modify a WriteBatch that has been committed`. This is probably due to a wrong use of batch — MVT KVM, Oct 18 '21 at 13:16
Can you follow this [StackOverFlow thread](https://stackoverflow.com/questions/48941311/batch-write-to-firebase-cloud-firestore)? — Prabir, Oct 19 '21 at 11:43
@Prabir if I create a the batch in the foreach, wouldn't that mean that I use 1 batch for 1 firestore set? And if so wouldn't that make the use of batches pointless? — MVT KVM, Oct 19 '21 at 11:55
Can you provide the error stack trace? I would suggest adding log lines at the start and end of each function, also before and after every await, to make sure that the code is executing in the right order as expected. — Prabir, Oct 20 '21 at 14:11
I've solved the problem. It was because I was using a `foreach` loop which is not async. This caused the batch to be commited before items were added. I am now using a `for of` — MVT KVM, Oct 21 '21 at 06:45

score 1 · Accepted Answer · edited Oct 21 '21 at 06:56

I solved my problem replacing the foreachloop with a for of loop. Here is an example:

if (plans.results.length != 0) {
    const batch = db.batch()
    for (const plan of plans.results) {
        const jiraKey = plan.planItem.self.split('/').pop()
        const projects = await db.collection('projects').where('jiraKey', '==', jiraKey).get()
        // ... 
        //set the batch with info
    }
    try {
        await batch.commit()
        await incrementOffsetTempo(db, 'tempo_plans')
        console.log("--- tempo_plans_done ---")
    } catch (error) {
        let key = "tempo_plans_error_sync"
        let message = "Impossible d'ajouter les plans. Erreur : " + error
        await notifications.addNotifications(db, key, message)
    }
}

Handle Deadline exceeded Firebase Cloud Functions

1 Answers1