0

I have a node.js app (my-app) that I'm setting Continuous Deployment for. The repository is hosted on Bitbucket and I'm creating CD with Bitbucket's Pipelines. The pipelines script deploys the app to Google Cloud Platform, to App Engine.

I am encountering a problem when I try to add a package (my-package) to my-app that is in another private repository on Bitbucket. Here is an part of my packages.json file:

"dependencies": {
    "my-package": "git+ssh://git@bitbucket.org:team-name/my-package.git",
}

With such linking I was able to:

  1. Add my PC's ssh key to repository to make sure npm install works locally
  2. Add pipelines ssh key to repository to make sure that npm install works inside pipelines environment

What I'm struggling with is the gcloud app deploy command inside the pipelines script. To my best knowledge, this command, when deploying a node.js app, runs npm install inside Google's environment. Such environment doesn't have access to my private repository with my-package and subsequently throws this error:

ERROR: (gcloud.app.deploy) Error Response: [9] Cloud build ##### status: FAILURE.
(...)
INFO     gen_package_lock npm install --package-lock-only
INFO     `gen_package_lock` stdout:
INFO     `gen_package_lock` had stderr output:
npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t ssh://git@bitbucket.org:team-name/my-package.git
npm ERR! 
npm ERR! Host key verification failed.
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR! 
npm ERR! exited with error code: 128

The obvious solution for me would be a possibility to generate an ssh key inside the Google's environment and granting access to the private repository.

Is it possible? If not - what other solutions could be applied here?

johnatann
  • 145
  • 11

1 Answers1

3

I was able to make it work and will happily share with you the steps that I’ve taken. I’ve posted a working example in my GitHub so you can clone it and try it out for yourself, do note that I will be deleting this after a week. I will be going through the whole process from scratch and I encourage you to do so, even if you already have your SSH keys set up.

  1. First and foremost, clone the GitHub repo and cd into it.
  2. Create a cloud KMS KeyRing and CryptoKey. Assuming you have the Cloud SDK installed, run gcloud kms keyrings create my-keyring --location=global to create a keyring. Next, create the CryptoKey by running gcloud kms keys create bit-key --location=global --keyring=my-keyring --purpose=encryption.
  3. Next, create an SSH key by running ssh-keygen -t rsa -b 4096 -C your_email@example.com. It will prompt you for a paraphrase and a file to save the key, just press each time.
  4. Configure your SSH key into Bitbucket. Go into bitbucket.org under "Bitbucket Settings" and then click "SSH Keys" in one of the options. Add a key (label it whichever name you decide) and paste the output you get from running cat ~/.ssh/id_rsa.pub into the key section.
  5. Take note of where your id_rsa file is located and get its path relative to your project folder. For example, because I did this reproduction in the Cloud Shell, my .ssh/id_rsa file’s path relative to my project folder is simply ../.ssh/id_rsa. Then encrypt the SSH key by running gcloud kms encrypt --plaintext-file=<RELATIVE_PATH_TO_.SSH/id_rsa> --ciphertext-file=./id_rsa.enc --location=global --keyring=my-keyring --key=bit-key. In my example, would simply be ../.ssh/id_rsa
  6. Grant your Cloud Build service account decrypt permission by first going into console.cloud.google.com/iam-admin and find the Cloud Build service account: its name is similar to @cloudbuild.gserviceaccount.com and grant it the App Engine Admin role (this is so that we are able to run gcloud app deploy from Cloud Build later on) then copy the email address of that @cloudbuild.gserviceaccount.com service account and go to console.cloud.google.com/security/kms and select the checkbox of ‘my-keyring’. On the panel to the right, you will see the permissions and there add a new member and paste the service account you just copied. Grant it the Cloud KMS > Cloud KMS CryptoKey Decrypter role.
  7. Create or update the known_hosts file by running ssh-keyscan -t rsa bitbucket.org > known_hosts
  8. Run gcloud builds submit --config=cloudbuild.yaml

As you can see, I have a private repository hosted on bitbucket called circular-structure-stringify. From the cloudbuild.yaml, you will see that the SSH key is first decrypted into a plaintext located in /root/.ssh/id_rsa, which is then used in the next step to set up the key with Bitbucket.

Next, we clone the private repository from Bitbucket into my container, followed by an npm install and gcloud app deploy. As you will notice, the dependency now lives in the same folder as our application, which is the same folder where the package.json file is located.

Finally, in your package.json, add the dependency in the dependencies property like such: dependencies: {“circular-structure-stringify”: “./circular-structure-stringify”}. Subsequently, you can import the module like you would in any npm packages: const CircularStructureStringify = require(‘circular-structure-stringify’) like you can see in the /routes/index.js file.

Previous submission (summary of steps detailed above)

To answer your question, yes, it is possible. You will have to use Cloud KMS 1 to interact with a private Bitbucket repository. There exists a documentation explaining the steps needed to access a private Github repository [2], but it must be adjusted slightly to make it work with Bitbucket.

Furthermore, when generating an SSH key, make sure to provide the -C “email@example.com” is specified. From past experiences, I’ve had issues with Bitbucket specifically if the key didn’t have this set upon creation (YMMV).. You can refer to this document [3] for step by step instructions.

Another solution would be to have your app hosted on the private repository and then mirror/clone that repository using Google Cloud Source Repositories [4], run npm install and deploy. As explained in this StackOverflow post, [5], you will have to create a cloudbuild.yaml file on the root folder (same folder the app.yaml file is located) :

steps:
# NPM install
- name: 'gcr.io/cloud-builders/npm'
  args: ['install']
#Test
- name: 'gcr.io/cloud-builders/npm'
  args: ['test']
#Deploy
- name: "gcr.io/cloud-builders/gcloud"
  args: ["app", "deploy"]

You will then have to mirror the private Bitbucket repository to Cloud Source Repository [4], create a Cloud Build Trigger to automate deployment when new code has been pushed to the repository, and then finally pushing the folder containing your application to the repository.

JKleinne
  • 1,270
  • 7
  • 14
  • thanks for the answer! I have tried proposed solution with using cloud build. I can indeed use an ssh key as it was described in [2] but running `- name: "gcr.io/cloud-builders/gcloud" args: ["app", "deploy"]` still creates same problem. My understanding is, that functionality of `gcloud app deploy` cannot be changed, and by default it will create it's own clean environment, remove `node_modules` folder and run `npm install`, which in my case - will result in an error. – johnatann Jul 30 '19 at 07:54
  • There should be a hidden .gcloudignore file in the directory which you can find by running “ls -lha”. Try removing the node_modules folder from the .gcloudignore file so that the folder will get uploaded as well when running gcloud app deploy. As such, it won’t be necessary to run npm install in the cloudbuild.yaml anymore, as long as npm install is ran before each deploy to ensure consistency. – JKleinne Jul 30 '19 at 19:39
  • Still, after running a `gcloud app deploy` it will remove `node_modules` and run `npm install` when deploying the app – johnatann Jul 31 '19 at 10:00
  • Running `gcloud app deploy` will not exclude uploading the node_modules/ folder if it is excluded from the hidden .gcloudignore file. If you don’t have that file try creating one in the root folder and append `!node_modules/` to explicitly include the node_modules/ folder when running `gcloud app deploy` – JKleinne Jul 31 '19 at 23:58
  • JKleinne, `google app deploy` will: 1. Upload your files, 2. Create deployment environment, 3. Download your files to that environment, 4. Remove `node_modules` in that environment, 5. Run `npm install` in that environment. I don't see a way to set ssh keys to be used inside that environment – johnatann Aug 01 '19 at 13:23
  • Can you provide a snippet of your .gcloudignore, cloudbuild.yaml and app.yaml file? What error message(s) do you get when deploying with a cloudbuild.yaml file along with a .gcloudignore file containing `!node_modules`? I’d suggest giving this article ( https://medium.com/google-cloud/continuous-delivery-in-google-cloud-platform-cloud-build-with-app-engine-8355d3a11ff5) a read as it is similar to what you are asking. – JKleinne Aug 02 '19 at 00:01
  • JKleinne, what I'm trying to say is that whatever setup you have, after running `gcloud app deploy`, Google, on their side, will remove `node_modules` folder and run `npm install` – johnatann Aug 02 '19 at 14:08
  • I added a detailed step-by-step on how this can be done in my previous answer after reproducing this myself. Let me know if any of the steps isn't clear. – JKleinne Aug 02 '19 at 23:30
  • Thank for the effort and improving the answer! I can see your solution indeed works – johnatann Aug 06 '19 at 14:17
  • @JKleinne Thank you for your step-by-step. Unfortunately your example does not exist any more in github and I face the same issue as johnatann. If I exclude `node_modules` in `.gcloudignore`, I get `INVALID_ARGUMENT: This deployment has too many files. New versions are limited to 10000 files for this app.` when deploying. – Treecj Sep 30 '19 at 14:51
  • @Treecj No worries, I'll take a look at the [thread](https://stackoverflow.com/questions/58169927/google-app-engine-deployment-with-private-git-repo-in-package-json) that you created earlier. – JKleinne Sep 30 '19 at 15:52
  • Do I need proceed all these steps if gcloud failed get not private repo and fail with same error? – CodeBy Apr 27 '22 at 13:29