0

I am new to cloud functions and a little unclear about the way they are "containerized" after they are written and deployed to my project.

I have two quite different sets of functions. One set deals with image storage and firebase, another deals with some time consuming computations. They two sets (lets call them A and B) of functions use different node modules and have no dependecies on each other, except they both use Firestore.

My question is wehther it matters if I put all the functions in a single VS Code project, or if I should split them up in separate projects? One question is on the deployment side? (It seems like you deploy all the functions in the project when you run firebase deploy changes, even if some of the functions haven't changed, but probably more important is whether or not functions which don't need sharp or other other image manipulation packages are "containerized" together with other functions which maybe need stats packages and math related packages, and does it make any difference how they are organized into projects?

I realize this question is high level and not about specific code, but its not so clear to me from the various resources what is the appropriate way to bundle these two sets of unrelated cloud functions to not waste a lot of unecessary loading once theya re deployed out to Firestore.

GGizmos
  • 3,443
  • 4
  • 27
  • 72
  • Linking these related questions: https://stackoverflow.com/a/58877253/3068190 and https://stackoverflow.com/a/66815121/3068190 – samthecodingman Jan 31 '22 at 03:36
  • And what's the point in putting them into different projects? – Martin Zeitler Jan 31 '22 at 04:54
  • By **project** do you mean Google Cloud Project or Visual Studio Project? – John Hanley Jan 31 '22 at 06:58
  • I mean Visual Studio Code projects. The reason I suspect putting that all the functions in a single project get loaded up each time you call a function is because only one of the functions is supposed to include app.initialize in its global scope. So I'm guessing that all the functions in a project and all of their respective nodemodules are loaded up into a single container on invoication. So if you have heavyweight node modules that are only used by one of the functions, they all end up being loaded into a fat container. – GGizmos Jan 31 '22 at 07:26
  • I actually have the same question and am still not clear on the answer reading through the replies. Did you come to a conclusion on this? – Dennis Ashford Jun 21 '22 at 22:13

2 Answers2

1

Visual studio code project is simply a way to package your code. You can create 2 folder in your project, one for each set of function with their own firebase configuration.

Only the source repository can be a constraint here, especially if 2 different teams work on the code base and each one doesn't need to see the code of the other set of functions

In addition, if you open a VS code project with the 2 set of functions, it will take more time to load them and to lint them.

On Google Cloud side, each functions are deployed in their own container. Of course, because the packaging engine (Buildpack) doesn't know, the whole code is added inside the container. When the app start, the whole code is loaded. More you have code, longer will be the init.

If you have segregate your set of functions code in different folder in your project, only the code for the set A will be embedded in the container of functions A, and same thing for B.

Now, of course, if you put all the functions at the same level and the functions doesn't use the same data, the same code and so on, it's:

  • The mess to understand which function do what
  • The mess in the container to load too much things

So, it's not a great code base design, but it's beyond the "Google Cloud" topic, and an engineering choice.

guillaume blaquiere
  • 66,369
  • 2
  • 47
  • 76
  • Thank you Guillaume. Perhaps to simplify, Lets say I have 3 cloud functions. 2 are very simple and just read and write firestore, and run constantly frequently. The 3rd function has heavy node module dependencies, needs a lot of memory and runs only occassionally. From what I can tell from the docs, in Firebase if I have 3 functions, normally all the functions are loaded in the same container for execution and firestore decides how many containers it needs to handle the load. – GGizmos Jan 31 '22 at 09:17
  • What I am trying to figure out if it is either a) possible or b) advisiable to somehow segregate the relatively lightweight and often used functions, from the heavyweight, in frenquently used functions, so the frequently used function and their containers are not burdened by one resource hogging function which is used only occassionally. – GGizmos Jan 31 '22 at 09:18
0

Initially I was really confused on GCP project vs VS Code IDE project...

On a question about how cloud functions are "grouped" into containers during deployment - I strongly believe that each cloud function "image" is "deployed" into its own dedicated and separate container in the GCP. I think Guillaume described it absolutely correctly. At the same time, the "source" code packed into an "image" - might have a lot of redundancies, and there may be plenty of resources, which are not used by the given cloud function. it may be a good idea to minimize that.

I also would like to suggest, that neither development nor deployment process should depend on the client side IDE, and ideally the deployment should not happen from the client machine at all, to eliminate any local configuration/version variability between different developers. If we work together - I may use vi, and you VS Code, and Guillaume - GoLand, for example. There should not be any difference in deployment, as the deployment process should take all code from (origin/remote) git repository, rather than from the local machine.

In terms of "packaging" - for every cloud function it may be useful to "logically" consolidate all required code (and other files), so that all required files are archived together on deployment, and pushed into a dedicated GCS bucket. And exclude from such "archives" any not used (not required) files. In that case we might have many "archives" - one per cloud function. The deployment process should redeploy only modified "archives", and don't touch unmodified cloud functions.

al-dann
  • 2,545
  • 1
  • 12
  • 22
  • OK, so the firebase docs are saying create a a project using 'firebase init' and select 'functions'. This creates a project structure with a single "functions" directory. But what I am reading here suggests I can create more than one functions directory in that project, each of which has the node modules it requires and then I can deploy each directory separately to firestore using firebase deploy in each directory. Now if I just figure out how to create multiple functions directory in the same vs code project . . . – GGizmos Jan 31 '22 at 18:57
  • I guess, you might like to consider cloud functions without firebase at all. One does not need to use firebase to develop, deploy and maintain GCP cloud functions. The way, how do that - using 'firebase init' and so on - may be the root of your question. – al-dann Feb 01 '22 at 09:16