0

I've managed to get a Cloud Task working. I've had no issues with the below code and when the task is called at a specific time, everything gets deleted. The issue that I can't wrap my around is what to do if any of the Promises fail and how to retry them because the task was successfully called but some or all of the Promises inside the task might fail.

I have 5 Promises

1- run a fan-out/atomic update to delete some values from two different database refs

2- run a Transaction to decrement a property at the users path

3- run a Transaction to decrement a property at the total-posts path

4- delete the video from Storage

5- delete the thumbnail from Storage

From my understanding both Transactions will retry on there own (29x) but the other three Promises I don't know how to handle. I've been reading different posts and looking at Youtube videos but I can't get the hang of this. Unless I'm misunderstanding Promise.all, it doesn't seem like a viable option because it's all or nothing, when one fails nothing else gets executed

const cors = require('cors')({origin: true});
const admin = require('firebase-admin');
admin.initializeApp({ ... });

exports.fireCloudTaskAtSpecificTime = functions.https.onRequest((request, response) => {

    return cors(request, response, () => {

        let userId = request.body.userId;
        let postId = request.body.postId;
        let videoId = request.body.videoId;
        let thumbnailId = request.body.thumbnailId;

        var updates = {};
        updates[`/posts/${postId}`] = null;
        updates[`/user-postIds/${userId}/${postId}`] = null;

        const runTransactionOnUserPostCt = admin.database().ref('users').child(userId).child('postCt');
        const runTransactionOnTotalPostCt = admin.database().ref('total-posts-count').child('postCt');

        const videoIdPath = 'videos/' + userId + '/' + videoId;
        const thumbnailIdPath = 'thumbnails/' + userId + '/' + thumbnailId;

        admin.database().ref('posts').child(postId).once('value', snap => {

            if (snap.exists()) {

                // 1. fan-out
                const root = admin.database().ref();
                return root.update(updates)
                .then(() => {

                    // 2. first Transaction
                    return runTransactionOnUserPostCt.set(admin.database.ServerValue.increment(-1));
                })
                .then(() => {

                    // 3. second Transaction
                    return runTransactionOnTotalPostCt.set(admin.database.ServerValue.increment(-1));
                })
                .then(() => {

                    // 4. delete video from Storage
                    const videoBucket = admin.storage().bucket();
                    return videoBucket.file(videoIdPath).delete()
                })
                .then(() => {

                    // 5. delete thumbnail from Storage
                    const thumbnailBucket = admin.storage().bucket();
                    return thumbnailBucket.file(thumbnailIdPath).delete()
                })
                .then(() => {

                    response.status(200).send('Ok - Everything Deleted');
                })
                .catch((error) => {

                    console.log('ERROR - Something Failed: ', error); // not sure what to do here or how to determine which one(s) failed
                });

            } else {

                response.status(200).send('Ok - Post No Longer Exists'); // user already deleted it
            }

        .catch((error) => {
            
            response.status(300).send('Task failed', error); // Only the HTTP code 2XX (from 200 to 299) are considered as a task completion and stops the retries. All other return code are considered as a failure and imply a retry.
        });
    });
});
Lance Samaria
  • 17,576
  • 18
  • 108
  • 256
  • This might be a good story for Google Workflows ... https://cloud.google.com/workflows they provide sequential execution of steps and, it something goes wrong in a step, appear to allow you to correct it and continue. Another thought might be Cloud Composer which, again, allows one to break ones work into a DAG of activities to execute. Another thought might be that you want the steps ALL executed EVENTUALLY without concern for order ... so fire of 5 tasks instead of one. Last, consider manually recording state in DataStore and handling what succeeds and what fails youself. – Kolban Oct 10 '21 at 15:06
  • @Kolban thanks for the reply. I'm going to look at the first 2 options. Firing off 5 tasks instead of one is an interesting option -do you have any SO links that shows one Cloud Task that fires off several handlers at once? Or do you mean 5 separate Tasks each with their own individual handler. That's an interesting idea but seems expensive. What do you mean "manually recording state in DataStore"? Do you have any links about that, that's new to me – Lance Samaria Oct 10 '21 at 15:18
  • Howdy Lance ... I did indeed mean 5 separate tasks fired off in parallel. If the outcome is that ALL tasks MUST complete and they are independent, then 5 tasks MUST start and 5 tasks MUST complete and if anyone of the tasks fail, you will know and they can be retried. On the last concept about using datastore ... I'm imaging you having ONE application that has code in it for 5 functions that are run one after another. When one succeeds, you update a database record saying that it succeeded. If one fails, you FAIL the whole GCP task and when it restarts, it knows what it has already done. – Kolban Oct 10 '21 at 18:59
  • Hi, thanks for the help. I've been thinking a lot about what you said about 5 separate tasks and it makes a TON of sense. 1 issue though, I looked around and I don't think that it's possible to fire off 5 separate tasks a once. I'm going to do more research on your datastore idea. Thanks for the help! – Lance Samaria Oct 10 '21 at 19:02

1 Answers1

0

Adding according to the comments of your question: You could also try with the Promise.allSettled() method that returns a promise that resolves after all the given promises have either been fulfilled or rejected, with an array of objects that each describes the outcome of each promise, respectively.

It is typically used when you have multiple asynchronous tasks that are not dependent on one another to complete successfully, or you'd always like to know the result of each promise.

In comparison, the Promise returned by Promise.all() may be more appropriate if the tasks are dependent on each other / if you'd like to immediately reject any of them.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled