22

I'm seeing this Cloud Build error when I try to deploy a Cloud Function:

"Step #2 - "analyzer": [31;1mERROR: [0mfailed to initialize cache: failed to create image cache: accessing cache image "us.gcr.io/MY_PROJECT/gcf/us-central1/SOME_KEY/cache:latest": failed to get OS from config file for image 'us.gcr.io/MY_PROJECT/gcf/us-central1/SOME_KEY/cache:latest'"

I'm able to build and emulate the cloud function locally, but I can't deploy it due to this error. I was able to deploy just fine until now. I've looked everywhere and I can't find any discussion about this. Anyone know what's going on here?

UPDATE: I deployed a new function 3 days ago and now I can't seem to deploy an update to it. I get the same error. I'm fairly sure this is happening due to the lifecycle rule I set up to ensure I don't keep storing images of functions: Firebase storage artifacts is huge and keeps increasing. This rule is important to keep around because I don't want to pay for unnecessary storage, but it seems like it might be the source of our problem here. Can someone from Google look into this?

Sameer Madan
  • 221
  • 2
  • 5
  • 1
    Can you share your language, the command line used and a minimal piece of code to reproduce the issue? – guillaume blaquiere Apr 30 '21 at 18:48
  • I'm using Typescript. I just created a new function and that deploys fine. I updated this function and the update also deployed fine. However, when I try to update any of the functions I already have, it fails on all of them. Every single one. I don't know how to repro it, but it seems like the image for my existing functions that's stored in Firebase is somehow unable to be accessed. Any logs I can pull that would help? – Sameer Madan Apr 30 '21 at 19:01
  • I'm using this command to deploy from CLI: firebase deploy --only "functions:myFunction" – Sameer Madan Apr 30 '21 at 19:08
  • I commented out the source code for one of my existing functions, deployed it, effectively deleting it, then uncommented the source code, redeployed it, effectively re-creating it. That solved my problem and I can update this function now. However, this strategy doesn't work for any function that is actually being used in production. I'll literally need to delete and re-create it. I'd love to understand what's causing this problem in the first place. – Sameer Madan Apr 30 '21 at 19:56
  • Delete and recreate function isn't a solution because you have a time with no function and thus a 404 on user side. The deployments should be consistent. Maybe Google support could help you – guillaume blaquiere Apr 30 '21 at 20:52
  • @SameerMadan - I had the exact same problem today. I've been deploying identical code to several stacks for 9 months. Followed your suggestion, commented out offending function (one that was failing continuously), deployed, added back in, deployed...all worked. Only thoughts as to what has changed: (1) I recently added LIFECYCLE (21 days) to my "artifacts" bucket to get of old deploy images, or (2) possibly different patch version of node (v14.15.2 vs. v14.15.4 ??). Thanks for following up on this thread! – Greg Fenton May 03 '21 at 05:16
  • 1
    @GregFenton please upvote this issuetracker: https://issuetracker.google.com/issues/186832976 – policenauts May 03 '21 at 17:25
  • @GregFenton I also have LIFECYCLE rules set up, but have had them now for months so while it might be related, this might be something new. I also tried with a Java runtime function and am getting the same error, so I don't think it's specific to Node. – policenauts May 03 '21 at 17:33

3 Answers3

19

I got the same error, even for code that deployed successfully before.

A workaround is to delete the Docker images for the failing Firebase functions inside Container Registry and re-deploying the functions. (The images will be re-created upon deploying.)

The error still occurs sporadically, so I suspect this may be a bug introduced in Firebase's deployment process. Thankfully for now, the workaround above resolves the issue every time the error comes up.

dkthehuman
  • 284
  • 1
  • 8
4

I also encountered the same problem, and solved it by deleting the images in the Container Registry of Firebase Project.

I made a Script at that time, and I'll put it here. The usage is as follows. Please use it if you like.

  1. Install the Google Cloud SDK.
  2. Download the Script
  3. Edit CONTAINER_REGISTRY to your registry name. For example: CONTAINER_REGISTRY=asia.gcr.io/project-name/gcf/asia-northeast1
  4. Grant execute permission. - $ chmod +x script.sh
  5. Execute it. - $ sh script.sh
  6. Deploy your functions.
Kichiemon
  • 51
  • 7
0

I'm having the same problem for the last few days and in contact with the support. I had the same log and in my case it wasn't connected to the artifacts because the artifacts rebuild themselves automatically on deploy (read below about a subtle case related to the artifacts and how to fix it), but deleting the functions and redeploying solved it for me.

Artifacts auto cleanup

Note that if the artifacts bucket is empty, then the problem is somewhere else.

But if it's not empty, what you can do to resolve any possible problems related to the artifacts auto cleanup, is to delete the whole "container" folder manually in the artifacts which should solve it. Then just redeploy again. Make sure not to delete the artifacts bucket itself!

Dough from firebase confirmed in the question you referring to that removing the artifacts content is safe.

So, here is how to delete it:

  1. go to the google cloud console, select your project -> storage -> browser https://console.cloud.google.com/storage/browser

  2. Select the "artifacts" bucket

  3. Choose "containers" and delete it

If the problem was here, it should work fine after that.

This happens because the deletion rule you refer to in your question checks the "last updated" timestamp of each file while on redeploy only some files are updated. So the next day the rule will delete some of the files while leaving the others which will lead to the inconsistent state of the bucket in this case. So you just remove everything manually.

vir us
  • 9,920
  • 6
  • 57
  • 66
  • This didn't solve my problem unfortunately. I didn't have anything in the artifacts bucket when I tried deploying an update this morning. – Sameer Madan May 04 '21 at 18:36
  • @SameerMadan that means it's not a problem with the artifacts (unless firebase changed something recently). Did you also tried to force delete all your functions and redeploy them? – vir us May 05 '21 at 08:13
  • "Did you also tried to force delete all your functions and redeploy them?" --> Yeah I did. That succeeded, but after a couple of days, I was again unable to deploy updates to these new functions. – Sameer Madan May 05 '21 at 17:30