3

I've got storage trigger function which resize and replace uploaded image into storage and then update URL in my database

    }).then(() => {
        console.log('Original file deleted', filePath)
        const logo = storageRef.file(JPEGFilePath)
        return logo.getSignedUrl({ action: 'read', expires: date })

        // const logo = storageRef.child(JPEGFilePath)
        // return logo.getDownloadURL()

        // return storageUrl.getDownloadURL(JPEGFilePath)
    }).then((url) => {
        const newRef = db.collection("user").doc(uid)
        return newRef.set({
            profile: { profileImg: url[0] }
        }, {
                merge: true
            })
    })

here is how I set expiry date

const d = new Date()
const date = new Date(d.setFullYear(d.getFullYear() + 200)).toString()

However the image expire in few weeks (roughly about 2 weeks). Does anyone know how to fix that? I have even played with getDownloadURL as you can see from commented code but that doesn't seems to work in trigger

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
delmin
  • 2,330
  • 5
  • 28
  • 54
  • That second code block looks like it creates a datestring for 200 years in the future. Where is your actual call to update the image expiration date? https://cloud.google.com/storage/docs/lifecycle#expirationtime says this can be set at a bucket level with an "Age" setting -- so why not just use that rather than programmatically setting it? – abelito May 25 '19 at 11:44
  • 1
    hi @abelito the date constant is used in **return logo.getSignedUrl({ action: 'read', expires: date })** . I have even tried to hard code it as a fixed date but the result was the same – delmin May 25 '19 at 11:46

2 Answers2

2

Ok so I have tried something but I have no idea if this will work or not so I'll come back in 2 weeks to mark my question as answered if it will work. For those with the same problem I'll try to recapitulate what I've done.

1/ Download the service account key from console. Here is the link

https://console.firebase.google.com/project/_/settings/serviceaccounts/adminsdk

2/ Save the downloaded JSON file in your function directory

3/ Include the key in your function storage. But be careful how you set the path to the file. Here is my question about it

https://stackoverflow.com/a/56407592/11486115

UPDATE

I just found mistake in my function. My URL was provided by cloud function by mistake (commented code)

Here is complete function

const {
db
} = require('../../admin')


const projectId = "YOUR-PROJECT-ID"
const { Storage } = require('@google-cloud/storage');
const storage = new Storage({ projectId: projectId ,keyFilename: 'PATH-TO-SERVICE-ACCOUNT'})

const os = require('os');
const fs = require('fs');
const path = require('path');

const spawn = require('child-process-promise').spawn

const JPEG_EXTENSION = '.jpg'

exports.handler = ((object) => {
const bucket = object.bucket;
const contentType = object.contentType;
const filePath = object.name
const JPEGFilePath = path.normalize(path.format({ dir: path.dirname(filePath), name: 'profileImg', ext: JPEG_EXTENSION }))
const destBucket = storage.bucket(bucket)
const tempFilePath = path.join(os.tmpdir(), path.basename(filePath))
const tempLocalJPEGFile = path.join(os.tmpdir(), path.basename(JPEGFilePath))
const metadata = {
    contentType: contentType
}
const uid = filePath.split("/").slice(1, 2).join("")
const d = new Date()
const date = new Date(d.setFullYear(d.getFullYear() + 200)).toString()

if (!object.contentType.startsWith('image/')) {
    return destBucket.file(filePath).delete().then(() => {
        console.log('File is not an image ', filePath, ' DELETED')
        return null
    });
}

if (object.metadata.modified) {
    console.log('Image processed')
    return null
}

return destBucket.file(filePath).download({
    destination: tempFilePath
})
    .then(() => {
        console.log('The file has been downloaded to', tempFilePath)
        return spawn('convert', [tempFilePath, '-resize', '100x100', tempLocalJPEGFile])
    }).then(() => {
        console.log('JPEG image created at', tempLocalJPEGFile)
        metadata.modified = true
        return destBucket.upload(tempLocalJPEGFile,
            {
                destination: JPEGFilePath,
                metadata: { metadata: metadata }
            })
    }).then(() => {
        console.log('JPEG image uploaded to Storage at', JPEGFilePath)
        return destBucket.file(filePath).delete()
    }).then(() => {
        console.log('Original file deleted', filePath)
        //const logo = storageRef.file(JPEGFilePath)
        const logo = destBucket.file(JPEGFilePath)
        return logo.getSignedUrl({ action: 'read', expires: date })
    }).then((url) => {
        const newRef = db.collection("user").doc(uid)
        return newRef.set({
            profile: { profileImg: url[0] }
        }, {
                merge: true
            })
    }).then(() => {
        fs.unlinkSync(tempFilePath);
        fs.unlinkSync(tempLocalJPEGFile)
        console.log(uid, 'user database updated ')
        return null
    })
})

I'm pretty confident that this will work now.

delmin
  • 2,330
  • 5
  • 28
  • 54
1

Per the following links:

https://stackoverflow.com/a/42959262/370321

https://cloud.google.com/nodejs/docs/reference/storage/2.5.x/File#getSignedPolicy

Not sure which version of @google/cloud-storage you're using, but assuming it's 2.5.x, it looks like any value you pass in the date field is passed into new Date(), so it looks like your code should work as I tried it in my dev tools. The only thing I can guess is it doesn't like that you want a file to live for 200 years.

Per the source code:

https://github.com/googleapis/nodejs-storage/blob/master/src/file.ts#L2358

Have you tried a shorter amount of time -- or formatting it in the dateform at mm-dd-yyyy ?

abelito
  • 1,094
  • 1
  • 7
  • 18
  • I have thought about to set shorter amount of time but then I found this link and it looks like I'm not the only one with that problem https://github.com/googleapis/nodejs-storage/issues/244 I was wandering if there is some workaround to that or perhaps make getDownloadURL to work in trigger – delmin May 25 '19 at 11:58
  • I haven't thought to set the date in mm-dd-yyyy format yet. I can try it but it'll take 2 weeks to find out. Actually I have tried that when I was hardcoding it I set it to 01-01-2100 as far as I can remember – delmin May 25 '19 at 12:02
  • 1
    So reading that thread you mentioned someone says this is a result of rotating keys on the service account that created the signed URL: https://github.com/googleapis/nodejs-storage/issues/244#issuecomment-404136388 This can happen if you use the default Google-provided service account which automatically rotates every so often. This is because changes to service accounts can invalidate signed URLs created by that service accounts. The work around appears to be specifying a specific service account's credentials, and then never updating that service account. I'd suggest this and try again! – abelito May 25 '19 at 12:05
  • thanks for pointing me there :) Now the question is where am I suppose to get the serviceAccount key. Where do I find it? – delmin May 25 '19 at 12:32
  • And that's where my knowledge ends unfortunately, though I may know in a few weeks as I'll likely have to do something similar. Best of luck though my friend! – abelito May 25 '19 at 12:36
  • 1
    You can read about service accounts in Google's documentation about [create & managing service accounts](https://cloud.google.com/iam/docs/creating-managing-service-accounts) and [getting a service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys#getting_a_service_account_key) documentation. – Noohone May 27 '19 at 13:38
  • @TasosV Thanks for the links... I will have to take a look at it this weekend and I will post a solution here if any. It is really weird that no one can give a proper solution here – delmin May 30 '19 at 14:23
  • The thing is that the only possible solution for you here is to create a new SignedURL for your object. Of course, you can do your objects public but that's a hard solution. – Noohone May 30 '19 at 15:48
  • @TasosV That is ridiculous. I just can not believe that google would give us an "image magic" (or whatever the name is it) for manipulating an image and then leave us with only signedURL where is no way to have that url long lived and create new signedURL after every time it expire? What is the point of setting expiry date? What is the point of "image magic"? What is the point of all of this then? – delmin Jun 01 '19 at 10:34