0

Cloud build uses worker pool of VM and that is not able to access my on-prem Compute Engine resources So, is there any way to run cloud build on my own VM or any solution for these?

LundinCast
  • 9,412
  • 4
  • 36
  • 48
Shashikumar KL
  • 1,007
  • 1
  • 10
  • 25

2 Answers2

2

While waiting for the custom worker-pool feature you mentioned in your previous question to become available to public, you can use the custom builder remote-builder.

You'll need to first build the builder image that you'll be able to use then in your Cloud Builds steps. When using the remote-builder image, the following will happen:

  1. A temporary SSH key will be created in your Container Builder workspace

  2. A instance will be launched with your configured flags

  3. The workpace will be copied to the remote instance

  4. Your command will be run inside that instance's workspace

  5. The workspace will be copied back to your Container Builder workspace

The build steps using this builder image will therefore run on a VM instance in your project's network and will be able to access other resources, provided your network configuration allows it.

Edit: The cos image used in the example cloudbuild.yaml file seems to include it so you'd be able to run it directly. In case you'd like to customize your instances with specific software, you have several options:

  • you can create an instance template (based on a custom image that includes the software or with a startup script that will install it at boot time) and specify that instance template in INSTANCE_ARGS in your cloudbuild.yaml.
  • you can use a standard image and just pass the startup script installing the software as INSTANCE_ARGS.
  • you can install it within a shell script executed in your build step.
LundinCast
  • 9,412
  • 4
  • 36
  • 48
  • can i run docker build inside remote-builders @LundinCast – Shashikumar KL Dec 06 '19 at 10:53
  • ```gcloud compute ssh --ssh-key-file=builder-key shashi@builder-ccb0b0c9-9749-416f-8ca8-0cb8ac4d9714 -- docker build --network=default -t quay.io/${PROJECT_ID}/imagename:${BRANCH_NAME} -f Dockerfile . Step #2: Pseudo-terminal will not be allocated because stdin is not a terminal. Step #2: Failed to add the host to the list of known hosts (/builder/home/.ssh/google_compute_known_hosts). Step #2: unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /home/shashi/Dockerfile: no such file or directory Step #2: + cleanup``` – Shashikumar KL Dec 09 '19 at 06:51
  • I want to run ***docker build --network=default -t quay.io/${PROJECT_ID}/image_name:${BRANCH_NAME} -f Dockerfile .*** command – Shashikumar KL Dec 09 '19 at 06:52
  • please give me some solution for this @LundinCast – Shashikumar KL Dec 09 '19 at 06:54
0

Why can't you just fix the access issue? You can configure cloud build to create build workers within your VPC within your cloud infrastructure:

See the following video which explain how this works: https://youtu.be/IUKCbq1WNWc?t=820

Hope this helps.

Parth Mehta
  • 1,869
  • 5
  • 15
  • This is currently still in EAP/Alpha actually, thus the workaround asked by the OP until it is officially released: https://cloud.google.com/sdk/gcloud/reference/alpha/builds/worker-pools/create – LundinCast Dec 06 '19 at 14:23
  • **Error says:** project is not whitelisted for this feature, please contact cloud-build-contact@google.com – Shashikumar KL Dec 07 '19 at 01:55