4

I am working on a project that uses Firebase Realtime Database and Firebase Storage.

First Part:

I have a function to upload the Pictures of users in javascript:


const firebaseConfig = {
    apiKey: FB_WEB_API_KEY,
    authDomain: FB_WEB_AUTH_DOMAIN,
    databaseURL: FB_WEB_DATABASE_URL,
    projectId: FB_WEB_PROJECT_ID,
    storageBucket: FB_WEB_STORAGE_BUCKET,
    appId: FB_WEB_APP_ID,
}

firebase = require('firebase/app')
require('firebase/functions')
require('firebase/database')
require('firebase/storage')
require('firebase/auth')
require('firebase/functions')
firebase.initializeApp(firebaseConfig)


export async function setUserPicture(pId, userId, picture) {
    if (picture !== '' && picture != null) {
        let promises = []

        const filesRef = firebase.storage().ref(`users/${pId}/${userId}`)

        filesRef.listAll().then(async folder => {
            for (let file of folder.items) {
                promises.push(filesRef.child(file.name).delete())
            }

            // Wait until all previous images are deleted
            await Promise.all(promises)

            let attachmentRef = firebase
                .storage()
                .ref(`users/${pId}/${userId}/${userId}@${Date.now()}`)

            attachmentRef.put(picture).then(_ => {
                attachmentRef.getDownloadURL().then(url => {
                    firebase
                        .database()
                        .ref(`users/${pId}/${userId}`)
                        .update({ photoURL: url })
                })
            })
        })
    }
}

In this function all images related to user's avatars are uploaded to a folder called users in Firebase Storage following this pattern:

'users/${pId}/${userId}/'       // Inside this path is placed the images

Before upload the new image, I delete all previous images from the specific folder in the storage, to be sure that there are no previous images.

Also when I upload the image, I take the downloadURL and save it in a field of the User data.

This are the packages installed for this:

"firebase": "^7.12.0",
"firebase-admin": "^8.11.0",
"firebase-functions": "^3.6.1",

Second Part:

For this project, I have some Cloud Functions to handle some data. In this case, I have a cloud function to save 2 resized images in the Storage, in the same folders where I have the original image of the avatar. This is to use proper sizes to improve the load time.

const functions = require('firebase-functions')
const admin = require('firebase-admin')
const { tmpdir } = require('os')
const { dirname, join } = require('path')
const sharp = require('sharp')
const fs = require('fs-extra')
const { uuid } = require('uuidv4')

exports.onUploadUserPicture = functions
    .runWith({ memory: '2GB', timeoutSeconds: 120 })
    .storage.object()
    .onFinalize(async object => {
        const bucket = admin.storage().bucket()
        const originalPath = object.name
        const originalFileName = originalPath.split('/').pop()
        const userId = originalFileName.split('@').shift()
        const bucketDir = dirname(originalPath)

        const workingDir = join(tmpdir(), 'resizing')
        const tmpPath = join(workingDir, 'origin.png')

        const metadata = {
            contentDisposition: object.contentDisposition,
            contentEncoding: object.contentEncoding,
            contentLanguage: object.contentLanguage,
            contentType: object.contentType,
            metadata: object.metadata || {},
        }

        metadata.metadata.resizedImage = true

        console.log(`Apply resizing to ${originalFileName}`)

        if (!originalPath.includes('users') || !object.contentType.includes('image')) {
            console.log(`This file is not a user picture. Exit!`)
            return false
        } else if (originalFileName.includes('_50x50')) {
            console.log(`Already processed: [ ${originalFileName} ] for size: [ 50 ]. Exit!`)
            return false
        } else if (originalFileName.includes('_300x300')) {
            console.log(`Already processed: [ ${originalFileName} ] for size: [ 300 ]. Exit!`)
            return false
        }

        await fs.ensureDir(workingDir)
        await bucket.file(originalPath).download({ destination: tmpPath })

        const sizes = [50, 300]

        const promises = sizes.map(async size => {
            console.log(`Resizing ${originalFileName} at size ${size}`)

            const newImgName = `${userId}@${Date.now()}_${size}x${size}`
            const imgPath = join(workingDir, newImgName)
            await sharp(tmpPath)
                .clone()
                .resize(size, size, { fit: 'inside', withoutEnlargement: true })
                .toFile(imgPath)

            console.log(`Just resized ${newImgName} at size ${size}`)

            // If the original image has a download token, add a
            // new token to the image being resized
            if (object.metadata.firebaseStorageDownloadTokens) {
                metadata.metadata.firebaseStorageDownloadTokens = uuid()
            }

            return bucket.upload(imgPath, {
                destination: join(bucketDir, newImgName),
                metadata: metadata,
            })
        })

        await Promise.all(promises)
        return fs.remove(workingDir)
    })

In this Cloud Function, firstly, I verify if it is user picture (based on the location of the original image) and if it is not a resized image (not have a _50x50 or _300x300)

Then, I download the original image, create both resized thumbnails, and after that upload to Storage.

I apply some metadata from the original image, a convention name to be unique and also generate a new storage download token for each image,

The PROBLEM

The real issue is that when I verify in Firebase Storage, sometimes, in a random way, the picture uploaded or that appear in the Storage is not correct. Some times is a picture that I uploaded before and even that supposed to be deleted previously by one of these functions.

Is there something missing in these functions??

Yulio Aleman Jimenez
  • 1,642
  • 3
  • 17
  • 33
  • Can you also include the imports of the first part? I see in some other threads that the npm firebase client library is not recommended instead google-cloud to upload objects. https://stackoverflow.com/questions/41352150/typeerror-firebase-storage-is-not-a-function https://stackoverflow.com/questions/39848132/upload-files-to-firebase-storage-using-node-js/39848966#39848966 – Antonio Ramirez Aug 11 '20 at 19:08
  • @AntonioRamirez Thanks for your help. I already updated the question with your request. I rewiew the links you shared, but the problem here is that is uploading incorrect objects (images), but it is still uploading something because I debugged this many times, even with the Firebase Console in the storage, opened in the browser. – Yulio Aleman Jimenez Aug 11 '20 at 20:41
  • Thank you. I have run some stress tests with the first part of your code. I basically uploaded 15 files using the firebase firestore UI and then ran your code using as input and event listener fileButton.addEventListener('change', function (e)... ). – Antonio Ramirez Aug 19 '20 at 21:36
  • I confirm the first part is deleting whatever is inside the users folder. By running some tests on my eventListener I found situations where the files were not deleted but this was because of the eventListener which was not triggering. This happened regardless using incognito mode. Can you please add the event listener of the first part of the code? – Antonio Ramirez Aug 19 '20 at 21:37
  • @AntonioRamirez Thanks for dedicating time to this. The function `setUserPicture` is receiving a picture as a blob, from an HTML Input. There is no more tricky code around this :). As I said before, the images are uploaded most of the time, the thing is that sometimes the image that appears in firebase storage is not the same that just uploaded :(. – Yulio Aleman Jimenez Aug 19 '20 at 22:11
  • I have run some tests integrating the second part of your code and I can see what you describe as “the image that appears in firebase storage is not the same that just uploaded”. From my side I always see the difference between the original image and the resized images. Can you confirm this difference? Or is it the other way around? – Antonio Ramirez Aug 31 '20 at 21:49
  • Adding some `console.log(new Date(),new Date().getMilliseconds());` before and after the await `Promise.all(promises)` of the first part of the code and comparing it with the time that appears when the storage function is triggered I see that the function is executed while the `promise.all(promises)` has not yet completed. Do you also see this? Have you tried comparing the timings that appear in the functions logs? – Antonio Ramirez Aug 31 '20 at 21:50
  • After doing some more tests with your code, I think the issue is related to the fact that your firebase function is its own event creator and the fact that I have not seen anywhere in your code if you clean your temporary directories. In this firebase example[1] they do it at the end with unlink method and in this other thread[2] they ask for ways to clean files in os.temp(). I see the files are deleted initially and the firebase function starts after it but I do no much visibility in the process order and variables values when you firebase function triggers itself. – Antonio Ramirez Sep 09 '20 at 16:43
  • [1] https://firebase.google.com/docs/functions/gcp-storage-events#example_image_transformation [2] https://stackoverflow.com/questions/43607942/cleaning-temporary-files-after-file-upload-with-restify – Antonio Ramirez Sep 09 '20 at 16:43

0 Answers0