28

So recently Firebase started charging for Cloud Functions container storage: https://firebase.google.com/pricing

No free usage $0.026/GB

I have deployed 2 functions several times (no more than 10 times, can't remember exact count, but this is still pretty low, IMO). Now I am already billed a small amount (fractions of a cent for now). So seems that if I deploy the functions another few dozens of times, I'll get close to a dollar, because old (and unused) containers are not deleted from the storage bucket.

Is there a way to safely delete outdated, not used containers to free some space? Well, it may seem that a few cents are not worth the time, but still, that's not what a free tier should be like.

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
ololoepepe
  • 979
  • 1
  • 10
  • 18

3 Answers3

12

I found the only robust solution to this ongoing issue (for now) is to periodically remove all of the artifact files (following Doug's instructions). As noted by others, removing some of the files can cause subsequent deploy errors (I experienced these).

IMPORTANT: Only delete the artifact files, NOT the folders as this can also cause issues.

You can do partial or full deploys as normal without any issues (it seems that the artifact files are only referenced during the build/deploy process).

Not ideal by a long shot, but at least reduces the storage usage to the minimum (until it starts accumulating again).

Edit: I have experimented with Lifecycle rules in the artifacts bucket to try and automate the clearing out of the container, but the parameters provided cannot guarantee that ALL will be cleared in one hit (which you need it to).

For convenience, you can see the artifacts bucket from within the Firebase Storage UI by selecting the "Add Bucket" option and importing the buckets from GCP.

D G
  • 489
  • 5
  • 9
8
  1. Go to the Cloud console
  2. Select "Cloud Storage -> Browser" from the products in the hamburger menu.
  3. You will see multiple storage buckets there. Simply dig in to the buckets that start with "artifacts" or end with "cloudbuild" and delete the old files (by date) that you don't want.
Doug Stevenson
  • 297,357
  • 32
  • 422
  • 441
  • 11
    Well... I saw the bucket, of course. If it was so easy, I would not ask this question. Have you noticed that files in that bucket are actually Docker image layer files? So there are multiple files corresponding to an image. You can not just delete one "old" file, as it may be an image layer reused in later builds. Again, there is no 1 to 1 relation between function version and file in the bucket. – ololoepepe Sep 15 '20 at 03:51
  • The dates of the files correspond to the dates of deployment. It should not be hard to figure out which ones are old enough to delete. – Doug Stevenson Sep 15 '20 at 03:56
  • 3
    That does not work like this. You may have a layer that is used in all consequent images, forever (theoretically). Please, see: https://docs.docker.com/storage/storagedriver/#images-and-layers as this topic is outside of the scope of my question. – ololoepepe Sep 15 '20 at 04:18
  • 1
    I think you have answered yourself. If there are layers that are used in all subsequent images, then it is almost impossible to delete them without having any impact on your deployment. Deleting previous layers could make longer your CFs deployment. – Joss Baron Sep 23 '20 at 13:38
  • 1
    Take a look into this question: https://stackoverflow.com/questions/59937542/can-i-delete-container-images-from-google-cloud-storage-artifacts-bucket – Joss Baron Sep 23 '20 at 13:43
  • 11
    My "Storage browser" in Cloud Console is empty, but my Firebase Console shows that my Storage is using 717MB with 1.8GB sent and 2.2k requests. I never used Storage (but Functions), just enabled it trying to found out what is happening. I've been charged a few cents for it. I am afraid that when the app goes to Prod this will charge me insanely and unduly. Google should REALLY fix it / give better info about. – Henrique Bruno Sep 25 '20 at 16:06
  • 3
    @DougStevenson aren't these files be used when google want to scale up the firebase functions? is it safe to add rule to delete those files? In my case file names are like `/containers/images/sha256:000235e7a1f1da4e3688b91e01158c6ee3b6e32b3b9e2e97ad7d424bf0954d12` – Reza Jan 16 '21 at 02:39
8

In case of Firebase Cloud Functions you can see from their documentation (lifecycle of a background function section):

When you update the function by deploying updated code, instances for older versions are cleaned up along with build artifacts in Cloud Storage and Container Registry, and replaced by new instances.

When you delete the function, all instances and zip archives are cleaned up, along with related build artifacts in Cloud Storage and Container Registry. The connection between the function and the event provider is removed.

It means that there is no need to manually cleanup and firebase deploy scripts are doing it automatically.

You should not remove build artifacts since cloud functions are scaling automatically and new instances are built from theese artifacts.

I don't really think that cost is nearly a problem since it's 0.026$/GB so you need very approximately about 76 functions to pay 1$ for their artifacts storage (I take approx size of 1 function's artifacts as 500mb). Also artifacts size should not grow up with every functions since it's basically size of dependencies which are more or less independent from number of deployed functions.

Georgii
  • 442
  • 4
  • 11