0

I have noticed through the Firebase Console that my gcf-sources buckets (one for each cloud function region) are taking a lot of storage space. (Around 2GB, when my functions folder is only 47Mb).

After digging into one of the buckets, I discovered that every function deployed has 2 folders (presumably for the latest and latest -1 versions). Inside these folders is a .zip file which contains the ENTIRE firebase functions folder. All of the script files, the full node_modules folder etc.

enter image description here

enter image description here

enter image description here

As you can imagine, this adds up very fast and 2GB is understandable.

My questions are:

  1. Why does every function need a zip file containing the entire functions directory?
  2. Can I safely delete all of this?
  3. How can I prevent this from happening in the future?

For a bit more context, I have done full and partial function deploys, but seem to have the same outcome either way.

Note - I came across this problem after following a previous question to clear up my artefacts folders: How to delete outdated Firebase Cloud function containers from GC Storage?

Chris Jacobs
  • 39
  • 4

1 Answers1

3

To understand why, you need to know the process. Your code is uploaded to Cloud Storage and a container is built based on your code (through Cloud Build and Buildpack). Each Cloud Functions version is independent and immutable, therefore there is no code sharing, each time you deploy a function, you deploy the WHOLE code inside that function and the container.

Can you delete those directory? Yes you can. After the deployment, the source file are useless on the bucket. You can set a lifecycle on the bucket if you want and to delete the file older than 1 day. Like that you keep your directory clean.

Be careful, I talk about ONLY the gcf source bucket!! Because the containers are stored in the artifact registry and if you delete the bucket registry you now longer able to get your container (and your code).

guillaume blaquiere
  • 66,369
  • 2
  • 47
  • 76
  • Thank you for explaining. I'm still a bit confused, the answers on the other SO post imply that the containers **can** be safely deleted (in fact Firebase's deployment process now removes them at the end). Is it correct that I can safely remove the containers (from the container console / let Firebase do it automatically) **and** delete the GCF source buckets? – Chris Jacobs Jan 12 '22 at 14:47
  • Yes and no... With Cloud Function v1, yes you can delete the container because only the latest one is deployed and loaded in cache in Cloud Functions. However, with Cloud Functions v2, it won't be the case. In fact, you will be able to perform rollback on previous version of your code and if you delete the container, it could be dramatic. I don't want to say YES today, to say NO in few months! – guillaume blaquiere Jan 12 '22 at 15:33
  • Ok so what I have done today is to delete the GCF source buckets **and** remove all the containers (I actually only had 2 despite having 10+ functions). I have now set a retention policy of 1 day on the GCF source bucket. I will defer to the deployment clean up process to manage the containers for now. This feels like a 'fresh start' and the functions are all currently working. Let's see what happens! Thank you – Chris Jacobs Jan 13 '22 at 06:39
  • Instead of "source files replication", my concern is on "binary replication" because I don't want to load all unnecessary libraries each time a function is called as it will impact the performance. @guillaumeblaquiere, do you know if Cloud Functions is smart enough to shake off those codes which are not necessary for a function to execute? Or it will blindly build everything in the functions folder and package it as a whole? – Cupid Chan May 09 '22 at 13:19
  • It mainly depends on your runtime environment and your language. In Golang, only the required libraries are compiled and stored in the binary. With Java, I know that a lib lazy loading exist, faster at startup time, but slower after (because of dynamic library loading). I don't really know the capabilities of Python and NodeJS. – guillaume blaquiere May 09 '22 at 15:00