Deploying firebase functions with local depencies using firebase CLI

Question

Setup

I have a monorepo setup with the following file structure:

├── functions
│   ├── src
│   └── package.json
├── shared
|   ├── dist
|   ├── src
|   └── package.json
├── frontend
|    └── ...
└── firebase.json

Approach 1 (failed)

./shared is holding TypeScript classes shared among the ./backend and ./frontend. Ideally, I want to reference the shared lib from the functions/package.json using a symlink to avoid that I have to re-install after every change to my shared code (where most of the functionality resides).

However, this does not work (neither using link:, nor an absolute file: path, nor an relative file: path)

// functions/package.json
  ...
  "dependencies": {
    "shared": "file:/home/boern/Desktop/wd/monorepo/shared"
    ...
  }

resulting into an error upon firebase deploy --only functions (error Package "shared" refers to a non-existing file '"home/boern/Desktop/wd/monorepo/shared"'). The library (despite being present in ./functions/node_modules/) was not transferred to the server?

Approach 2 (failed)

Also, setting "functions": {"ignore": []} in firebase.json did not help.

Approach 4 (works, but lacks requirement a) see Goal)

The only thing that DID work, was a proposal by adevine on Github:

// functions/package.json
  ...
  "scripts": {
     ...
    "preinstall": "if [ -d ../shared ]; then npm pack ../shared; fi"
  },
  "dependencies": {
    "shared": "file:./bbshared-1.0.0.tgz"
    ...
  }

Goal

Can someone point out a way to reference a local library in a way that a) ./functions always uses an up-to-date version during development and b) deployment using the stock Firebase CLI succeeds (and not, e.g. using firelink)? Or is this simply not supported yet?

Answer 1

To answer your question, monorepos are indeed not supported yet by the Firebase tools deploy command / pipeline.

I have developed a generic solution and wrote an article about it. Below is an excerpt with the relevant parts, but here is a link to the full article in case you are interested.

The problem with Firebase

When deploying to Firebase it wants to upload a folder just like a traditional single package repository, containing the source files together with a manifest file declaring its external dependencies. After receiving the files in its cloud deployment pipeline, it then detects the package manager and runs an install and build.

In a monorepo, and especially a private one, your Firebase code typically depend on one or more shared packages from the same repository, for which you have no desire to publish them anywhere.

Once Firebase tries to look up those dependencies in the cloud they can not be found and deployment fails.

Hacking your way out

Using a bundler

In order to solve this you could try to use a bundler like Webpack to combine your Firebase code with the shared packages code and then remove those packages from the package.json manifest that is being sent to Firebase, so it doesn’t know these packages even existed.

Unfortunately, this strategy quickly becomes problematic…

If the shared packages themselves do not bundle all of their dependencies in their output, Firebase doesn’t know what the shared code depends on, because you are not including or installing those manifests.

You could try to bundle everything then, but if your shared package depends on things your Firebase package also depends on, you now have one part of your code running an internally bundled copy of a dependency and the other part using that same dependency from a different location installed by the package manager.

Also, some libraries really don’t like to be bundled, and in my experience that includes the Firebase and Google client libraries. You will quickly find yourself trying to externalize things via the bundler settings in order get thing to work.

And even if you managed to make all this work, you are probably creating large bundles which could then lead to problems with the cold-start times of your cloud functions.

Not exactly a reliable or scalable solution.

Packing and linking local dependencies

An arguably more elegant approach involves packing the local dependencies into a tarball (similar to how a package would be published to NPM), and copying the results to the build output before linking them in an altered manifest file.

This could work quite nicely, as it basically resembles how your Firebase code would have worked if these packages were installed from an external domain.

Whether you’re doing this manually, or write a shell script to handle things, it still feels very cumbersome and fragile to me, but I think it is a viable workaround if your local dependencies are simple.

However, this approach quickly becomes hairy once you have shared packages depending on other shared packages, because then you’ll have have multiple levels of things to pack and adapt.

A solution

I spent a lot of hours trying out the different workarounds, before I realized what a convenient solution could look like. Initially, I was going for something specific to PNPM, but while working on it I realized that support for other package managers shouldn't be that difficult.

It feels good to be able to contribute something useful to the community, and so I have created isolate-package. The name is generic because it doesn’t contain anything specific to Firebase, although I currently don’t know of any other use-cases for isolated output.

It takes a similar approach to what is described earlier in packing and linking dependencies, but does so in a more sophisticated way. It is designed to handle different setups and package managers and it completely hides the complexity from the user.

The isolate binary it exposes can simply be added to the Firebase predeploy hook, and that’s pretty much it!

This also allows you to deploy to Firebase from multiple different packages and keep the configuration co-located instead of littering the monorepo root directory.

It should be zero-config for the vast majority of use-cases, and is designed to be compatible with all package managers.

Answer 2

Here's my workaround to make approach 4 work:

rm -rf ./node_modules
yarn cache clean # THIS IS IMPORTANT
yarn install

Run this from the ./functions folder