How to upload a multiple files on firebase storage and get the access urls and metadata as as json or a csv file?

Question

As an android developer this is a problem that I am stumped with.

I have about 5000 mp3 files that I want to add to my firebase storage in different folders. Till now I have been adding the files from the browser and simply copying and pasting the urls in the firestore and using them in my android project.

Now that I need to upload so many files how will I be able to get

1. The urls of the files This is really important I need to find out a way to get the urls of the uploaded files, I don't want to end up to do this manually.

2. Metadata of the files, just the duration actually. If this is doable with some plugin service with firebase then it will be really helpful or some third party application that I can run on my computer.

Please help.

Answer 1

One possible approach is to use a Cloud Function to generate a signed URL and get the file's metadata and store these info in a Firestore document.

The following Cloud Function code creates, for each file uploaded to the default bucket, a Firestore document with an auto-generated ID and a field fileName that contains the file name.

exports.saveFileDetails = functions.storage.object().onFinalize(async object => {

    try {
        const bucket = admin.storage().bucket(object.bucket);
        const file = bucket.file(object.name);

        const signedURLconfig = { action: 'read', expires: '08-12-2035' };

        const signedURLArray = await file.getSignedUrl(signedURLconfig);
        const url = signedURLArray[0];

        const metadata = await file.getMetadata();

        return admin.firestore().collection('...').add({ signedURL: signedURLArray[0], metadata: metadata[0], fileName: object.name });

    } catch (error) {
        console.log(error);
        return null;
    }

});

---

Note that you could use the file name value as the ID of the Firestore document, if you are sure it follows the constraints on document IDs.

Answer 2

To do this effectively, you can create a node app with the admin-sdk that will read the file structure of a specified folder and records the output as each file is processed.

• Get started with the admin-sdk
• Loop through files and process them
• Storage with the Admin sdk

NPM packages to read video length

• Option 1
• Option 2

NPM packages to manage CSV formatting

• Option 1
• Option 2