Google Cloud Builds, How to speed up build time of application(npm packages)

Question

My web application runs in Google Cloud Platform, lately I realized that application's build time takes really long, especially when you are testing a feature, let say refresh the page, you can see that it takes really long to start application. What I am looking for is how to speed up this process. I am using docker images to build on google cloud registry. I do not want to re-build all npm packages every time, when there is an update in some npm packages then I want to re-build application.

here is the my cloudbuild.yaml file for polymer

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: [ 'build', '-t', 'gcr.io/$PROJECT_ID/myapp-polymer', '.' ]
images:
- 'gcr.io/$PROJECT_ID/myapp-polymer'

then here is the my main cloudbuild.yaml file

steps:
- name: 'gcr.io/cloud-builders/npm'
  args: ['--prefix', 'myapp','install']
- name: 'gcr.io/cloud-builders/npm'
  args: ['--prefix', 'myapp/functions', 'install']
- name: 'gcr.io/$PROJECT_ID/myapp-polymer'
  args: ['cd', 'myapp']
- name: 'gcr.io/$PROJECT_ID/myapp-polymer'
  args: ['build']

I read Google Cloud API especially section "Speeding up your Builds" (https://cloud.google.com/cloud-build/docs/speeding-up-builds)

I think it just cache build images and using it. Are there any way that I can cache specifically npm packages, dependencies etc in Google Cloud so that I do not have to build every time whole application? My main goal is to reduce build time, speed up build process.

Thanks!

I'd save the `node_modules/` contents after the build to Cloud Storage - and read from there before next build. There is a similar third-party example online, but not `npm` specific. Optimizing these things is unfortunately a very manual job with GCP. https://github.com/GoogleCloudPlatform/cloud-builders/issues/266#issuecomment-384980135 — akauppi, Oct 08 '22 at 18:25

score 7 · Accepted Answer · answered Nov 24 '19 at 01:53

Have you tried Kaniko? It can save the cache in gcr.io and and if you have built your Dockerfile with right steps (see https://cloud.google.com/blog/products/gcp/7-best-practices-for-building-containers), it should save you a lot of time. Here is the cloudbuild.yaml example:

- name: 'gcr.io/kaniko-project/executor:latest'
  args:
  - --destination=gcr.io/$PROJECT_ID/image
  - --cache=true
  - --cache-ttl=XXh

More info: https://cloud.google.com/cloud-build/docs/kaniko-cache

score 1 · Answer 2 · answered Nov 08 '18 at 15:27

1

You could use docker.

Put your npm application into docker. Then you can push your docker image (upload some layers of your docker image) to a cloud registry, e.g. gcr (Google Cloud Registry). Before your build step, you could pull your image from gcr.

This is what the build step would approximately look like:

- name: 'gcr.io/cloud-builders/docker'
  args: [
          'build',
          '-t', 'test_image',
          '-f', 'Dockerfile',
          '.'
        ]
  id: 'build_test_image'

answered Nov 08 '18 at 15:27

markus-hinsche

1,372
15
26

Hello @markushinsche, Thanks for respond! I actually located my application in docker, what I wan to try that before build step in main googlebuild.yaml, I want to compare package.json files. If there is no update, then I do not want to " npm install " all package again. But I do not know how to compare package.json files outside of docker and inside of docker. I want to run my javascript file and send it two package.json check if they are equal then I want to create image npm i cache or npm install - depends on conditional result. In docker file how to RUN if else RUN commands :( – Erman Tatar Nov 08 '18 at 20:04

score 0 · Answer 3 · answered Mar 29 '23 at 16:08

Google Cloud uses vCPUs for everything so, if billing is no issue, then you can upgrade the CPU used to run the builds by updating cloudbuild.yaml with the following:

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/my-project/image1', '.']
options:
  machineType: 'E2_HIGHCPU_8' # <-- insert upgraded CPU type here

Source: https://cloud.google.com/build/docs/optimize-builds/increase-vcpu-for-builds

Google Cloud Builds, How to speed up build time of application(npm packages)

3 Answers3

Linked