Jenkins - Git Submodule Credentials Different from Parent Repo

Question

Background

Jenkins is being used to build an artifact from a Git repo that has a Git submodule. The submodule(s) are not in the same repo or even at the same endpoint as the parent project. The problem is that parent repo checks out fine because the credential, ssh key A, is associated with the main/parent repo but, not surprisingly, fails on the submodule because the credential, ssh key B, is not associated with the repo from Jenkins' point of view.

It's surprising that Jenkins does not have better out-of-the-box support for Git submodules; time to contribute.

Questions

1. Is there a way to store multiple credentials for a single Git repo?
2. If the answer is no, why does Jenkins, under Advanced sub-modules behaviours provide the option to Use credentials from default remote of parent repository as shown below?
3. What other approach is there for dealing with Jenkins and Git submodules with different credentials?

System Info

Running Jenkins on Docker Machine (Locally)

Running Jenkins on CentOS (Production)

Jenkins Version: 2.60.2 (both)

Git Plugin Version: 3.6.4 (both)

Answer 1

1. No - the git plugin supports only a single key (as of this writing)
2. Per the git plugin docs, the default behavior is to not use credentials at all for submodules. The boolean flag tells the plugin to instead use the parent credential. The git plugin it uses underneath does not support the idea of using any other credential.
3. This was very difficult to find but I was faced with the same issue with my Github project - it used modules and Github does not allow the same deploy key to be used across projects.

To work around this, I first set up a github app, installed it, and assigned the project and its submodules. This app is capable of generating access keys which can be used for cloning - so I used these credentials to clone. I disabled submodules in the git plugin behavior, and used native git command to update the submodule URL to use a username and password before running submodule init and submodule update to clone the submodules.

steps {
    withCredentials([usernamePassword(credentialsId: 'github-app-credentials',
                                usernameVariable: 'GITHUB_APP',
                                passwordVariable: 'GITHUB_ACCESS_TOKEN')]) {
        checkout ([
            $class: 'GitSCM',
            userRemoteConfigs: [[
                credentialsId: '',
            url: "https://x-access-token:$GITHUB_ACCESS_TOKEN@github.com/<ORG>/<PROJECT>.git"
            ]],
            branches: [[ name: '*/main' ]],
            extensions: [[
                $class: 'SubmoduleOption', 
                disableSubmodules: true
            ]]
        ])
        sh '''
            git config --file=.gitmodules submodule.SUBMODULE_A.url https://x-access-token:$GITHUB_ACCESS_TOKEN@github.com/<ORG>/<SUBMODULE_A>.git
            git config --file=.gitmodules submodule.SUBMODULE_B.url https://x-access-token:$GITHUB_ACCESS_TOKEN@github.com/<ORG>/<SUBMODULE_B>.git
            git submodule init update
            git restore .gitmodules

            # rest of build
        '''
    }
}

This flags a security issue in Jenkins as the GITHUB_ACCESS_TOKEN is being interpolated in a string in order to pass it to the git URL. I don't believe this is fixable as the checkout command will not reference any environment variable. However, it looks like these access tokens are ephemeral so it is somewhat mitigated.

I believe you can also use deploy keys by being cute with the GIT_SSH_COMMAND environment variable, though this is left as an exercise to the reader.

The next time someone suggests using git submodule, please hit them for me.

Answer 2

Yes this is easily done. You can create pub/private key pair and set it up as a github deploy key (if you are using git hub if not, then as a key set in whatever you do use). You can add that as a Jenkins credential (provided you have the credentials plugin installed).

Username: git@github.com
Private Key:  the private key for that key set
Passphrase:  whatever passphrase you used
ID: aws-jenkins-github-deploykey (just an example name)
Description: some useful text

The ID maps to the credentialsId below

checkout changelog: true, poll: false, scm: [$class: 'GitSCM',
     branches: [[name: "branch name, commit sha, or tag/tagname" ]],
     userRemoteConfigs: [[
     credentialsId: 'aws-jenkins-github-deploykey',
     url: 'git@github.com:myorg/myrepo.git']]]

When this Pipeline code runs, it checks out the repo at the branch, commit, etc to the working directory. You can also specify the directory.

So you can use this to checkout a bunch of repos and use specific branches for them.