How can I deploy a GAE service with a local npm dependency?

Question

I'm new to Google Cloud, and can't seem to figure out how to deploy a Google App Engine service that has a sibling local dependency. I have a project structure like this (my stack is TypeScript, nestJS, React):

-frontend
    app.yaml
    package.json
-backend
    app.yaml
    package.json
-common
    package.json
dispatch.yaml

The frontend package deploys static code to the default service, and the backend package deploys to an api service. The common package contains some code that I want to share between the front and backend. They include it in their package.json files like this:

dependencies: {
    common: "file:../common"
}

This structure works fine locally. The problem is that when I deploy the backend, npm install fails, because it can't find the common package. This make sense, since I understand that it's only going to upload the contents of backend to that service. But what is the right way to achieve this?

I suppose I could just deploy one service that contains all of the code, and have my top-level package.json delegate npm start to the backend's package.json. But that only works because my frontend is fully static, so I only have one package that needs npm start to get called. It seems like there should be a better way to handle this, because that approach would break down if I had two separate backends that each needed their own npm start.

I'm guessing I'm thinking about some aspect of this in a fundamentally wrong way, but I need help figuring out what that is.

score 1 · Answer 1 · answered Apr 19 '20 at 23:30

1

I’m not familiar with GAE, but my solution for sharing code like this is by packaging the shared code as an NPM package and importing it in both the server and client apps

If you choose not to make your shared package(s) public, you can still install directly from a git repository. Be warned that this may complicate your deployment since you’ll have to configure SSH keys to allow cloning of your private repository.

answered Apr 19 '20 at 23:30

Kryten

15,230
6
45
68

Could you provide an example of how you import an NPM package on the client side? On the server side you would just require it. – ultraGentle Apr 30 '20 at 13:24
For client side I use [Webpack](https://webpack.js.org/) to bundle all the dependencies. There are about a million tutorials out there on the subject. – Kryten Apr 30 '20 at 15:57
Okay, so it's not a direct import, like via a ` – ultraGentle Apr 30 '20 at 15:59

LundinCast · Accepted Answer · 2020-04-30T14:12:27.010

The thing here is that you define 2 App Engine services, frontend and backend, that will be packaged independently and installed on different instances, they'll never co-exist on a single instance. So both will need to include the common package.

When you run gcloud app deploy, the folder containing the app.yaml file for that service is considered the root folder and files and folder up in the tree won't be deployed, as you mentioned.

I understand that from a development point of view it makes sense to have a single common package shared by both services, since it avoids duplicating code. One way to manage this is to use Cloud Build to create a build pipeline that will handle incorporating this common code into both services and deploy them separately. For example, something like this:

steps:
- name: ubuntu
  id: 'copy-file'
  args:
  - '-c'
  - |
        cp ./common/package.json frontend/ && cp ./common/package.json backend/
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['app', 'deploy']
  dir: 'frontend/'
  timeout: '1600s'
  waitFor: ['copy-file']
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['app', 'deploy']
  dir: 'backend/'
  timeout: '1600s'
  waitFor: ['copy-file']

The first step will copy the common package to both directories (you'll need to update your common dependency path in your package.json since it'll now be in the same directory). The next 2 steps will run in parallel (both wait for the first step to finish) and deploy each service separately (note the dir parameter).

You can then deploy your services by running the following command in the root directory:

gcloud builds submit

Note that this will always deploy both services.

If you'd rather like to be able to deploy one service and not the other, you could define 2 cloudbuilds files like so:

cloudbuild-frontend.yaml:

steps:
- name: ubuntu
  args:
  - '-c'
  - |
        cp ./common/package.json frontend/
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['app', 'deploy']
  dir: 'frontend/'
  timeout: '1600s'

cloudbuild-backend.yaml:

steps:
- name: ubuntu
  args:
  - '-c'
  - |
        cp ./common/package.json backend/
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['app', 'deploy']
  dir: 'backend/'
  timeout: '1600s'

You'd end up with a tree like this:

-frontend
    app.yaml
    package.json
-backend
    app.yaml
    package.json
-common
    package.json
cloudbuild-frontend.yaml
cloudbuild-backend.yaml
dispatch.yaml

You'd then be able to deploy one service or the other by running either gcloud builds submit --config=cloudbuild-frontend.yaml or gcloud builds submit --config=cloudbuild-backend.yaml

score 1 · Answer 3 · answered Apr 30 '20 at 16:15

I had a similar situation with Firebase, where I had code I wanted to share between the front and the back, without having to maintain changes in two places.

Since you're using Typescript, you can simply use tsc, the Typescript Compiler, to output your common file to both your front- and back-end folders, by configuring the outputs in your tsconfig.json file. https://www.typescriptlang.org/docs/handbook/tsconfig-json.html

For the front, you'd output it as a .js file, and simply load it as <script src='myCommonFile.js>. In the back, you'd require or import it.

Then the trick is to write the code in such a way that it can be used in both browser and node environments. Quoting the code from this SO post:

(function(exports){

    // Your code goes here. For example: 
   exports.test = function(){
        return 'hello world'
    };

})(typeof exports === 'undefined'? this.mymodule = {} : exports);

Thus, if exports is undefined, you must be in the browser, in which case mymodule is declared on the window (i.e., this). Or, if exports is defined, it's in a node context, in which case you can just var mymodule = require('mymodule'). And in either environment, you can then use it as mymodule.test(). Nifty!

How can I deploy a GAE service with a local npm dependency?

3 Answers3