How to use pnpm workspace protocol and deploy Firebase functions?

Question

I have a pnpm monorepo and I use the workspace: protocol to add my shared folder as a local package.

The workspace: protocol is not recognized by Firebase when deploying functions and I have the following error:

Build failed: npm ERR! code EUNSUPPORTEDPROTOCOL
npm ERR! Unsupported URL Type "workspace:": workspace:../shared
npm ERR! A complete log of this run can be found in:
npm ERR! /www-data-home/.npm/_logs/2023-03-19T21_44_16_828Z-debug-0.log; Error ID: b0ba1f57

How do I deploy my Firebase functions using pnpm workspace?

Answer 1

This question is basically a duplicate of Deploying firebase functions with local depencies using firebase CLI

The PNPM workspace protocol is not supported, because the Firebase deploy command doesn't understand monorepos in general. You would face the same problem with Yarn or NPM based monorepos.

I have developed a generic solution for this and wrote an article about it. Below is an excerpt so that you have the full answer, but here is a link to the full article

The problem with Firebase

When deploying to Firebase it wants to upload a folder just like a traditional single package repository, containing the source files together with a manifest file declaring its external dependencies. After receiving the files in its cloud deployment pipeline, it then detects the package manager and runs an install and build.

In a monorepo, and especially a private one, your Firebase code typically depend on one or more shared packages from the same repository, for which you have no desire to publish them anywhere.

Once Firebase tries to look up those dependencies in the cloud they can not be found and deployment fails.

Hacking your way out

Using a bundler

In order to solve this you could try to use a bundler like Webpack to combine your Firebase code with the shared packages code and then remove those packages from the package.json manifest that is being sent to Firebase, so it doesn’t know these packages even existed.

Unfortunately, this strategy quickly becomes problematic…

If the shared packages themselves do not bundle all of their dependencies in their output, Firebase doesn’t know what the shared code depends on, because you are not including or installing those manifests.

You could try to bundle everything then, but if your shared package depends on things your Firebase package also depends on, you now have one part of your code running an internally bundled copy of a dependency and the other part using that same dependency from a different location installed by the package manager.

Also, some libraries really don’t like to be bundled, and in my experience that includes the Firebase and Google client libraries. You will quickly find yourself trying to externalize things via the bundler settings in order get thing to work.

And even if you managed to make all this work, you are probably creating large bundles which could then lead to problems with the cold-start times of your cloud functions.

Not exactly a reliable or scalable solution.

Packing and linking local dependencies

An arguably more elegant approach involves packing the local dependencies into a tarball (similar to how a package would be published to NPM), and copying the results to the build output before linking them in an altered manifest file.

This could work quite nicely, as it basically resembles how your Firebase code would have worked if these packages were installed from an external domain.

Whether you’re doing this manually, or write a shell script to handle things, it still feels very cumbersome and fragile to me, but I think it is a viable workaround if your local dependencies are simple.

However, this approach quickly becomes hairy once you have shared packages depending on other shared packages, because then you’ll have have multiple levels of things to pack and adapt.

My solution

I have created isolate-package. The name is generic because it doesn’t contain anything specific to Firebase, although I currently don’t know of any other use-cases for isolated output.

It takes a similar approach to what is described earlier in packing and linking dependencies, but does so in a more sophisticated way. It is designed to handle different setups and package managers and it completely hides the complexity from the user.

The isolate binary it exposes can simply be added to the Firebase predeploy hook, and that’s pretty much it!

This also allows you to deploy to Firebase from multiple different packages and keep the configuration co-located instead of littering the monorepo root directory.

It should be zero-config for the vast majority of use-cases, and is designed to be compatible with all package managers.

Answer 2

1. functions/package.json

{
  "private": true,
  "name": "functions",
  "main": "dist/index.js",
  "scripts": {
    "dev": "tsc --watch",
    "build": "tsc",
    "pre-deploy": "node pre-deploy.js",
    "post-deploy": "node post-deploy.js"
  },
  "dependencies": {
    "firebase-admin": "11.5.0",
    "firebase-functions": "4.2.1",
    "shared": "workspace:../shared"
  },
  "devDependencies": {
    "typescript": "4.9.5"
  },
  "engines": {
    "node": "18",
    "pnpm": "7"
  }
}

2. functions/pre-deploy.js

#!/usr/bin/env node

const { existsSync, copyFileSync, readFileSync, writeFileSync } = require('fs')

const packagePath = './package.json'
const packageCopyPath = './package-copy.json'
const pnpmWorkspaceRegex = /workspace:/gi

// Abort if "package-copy.json" exists
if (existsSync(packageCopyPath)) {
  console.error(`"${packageCopyPath}" exists, previous deployment probably failed.`)
  return
}

// Copy "package.json" file
copyFileSync(packagePath, packageCopyPath)

// Read "package.json" file and replace "workspace:" to "file:" protocol
const packageBuffer = readFileSync(packagePath)
const packageContent = packageBuffer.toString()
const packageNewContent = packageContent.replace(pnpmWorkspaceRegex, 'file:')

writeFileSync(packagePath, packageNewContent)

3. functions/post-deploy.js

#!/usr/bin/env node

const { rmSync, renameSync } = require('fs')

const packagePath = './package.json'
const packageCopyPath = './package-copy.json'

// Restore original "package.json" file with "workspace:" protocol
rmSync(packagePath)
renameSync(packageCopyPath, packagePath)

4. firebase.json

{
  "functions": {
    "source": "functions",
    "codebase": "default",
    "predeploy": [
      "pnpm --filter functions run build",
      "pnpm --filter functions run pre-deploy"
    ],
    "postdeploy": [
      "pnpm --filter functions run post-deploy"
    ],
    "ignore": [
      ".git",
      "node_modules",
      "firebase-debug.log",
      "firebase-debug.*.log"
    ]
  }
}

The steps above assume that the pnpm-workspace.yaml file contains the following information:

packages:
  - 'functions'
  - 'shared'