141

I'd like for Jenkins to automagically fetch data from my private repository hosted on Github. But I have no idea how to accomplish that task.. Tried the documentation, generating ssh-key for jenkins user and all what I can see is: "unable to clone the repo". I've checked URLs - they are valid.

Any clues, maybe you know some docs/blogs/whatever which are describing this kind of stuff?

ThomasW
  • 16,981
  • 4
  • 79
  • 106
bx2
  • 6,356
  • 5
  • 36
  • 40
  • I answer this a similar question, you can see the answer in the link below: [jenkins & GitHub](http://stackoverflow.com/questions/6515039/jenkins-git-permission-denied-publickey/8143432#8143432) – iDeasTouch Nov 15 '11 at 21:39

7 Answers7

140

Perhaps GitHub's support for deploy keys is what you're looking for? To quote that page:

When should I use a deploy key?

Simple, when you have a server that needs pull access to a single private repo. This key is attached directly to the repository instead of to a personal user account.

If that's what you're already trying and it doesn't work, you might want to update your question with more details of the URLs being used, the names and location of the key files, etc.

Now for the technical part: How to use your SSH key with Jenkins?

If you have, say, a jenkins unix user, you can store your deploy key in ~/.ssh/id_rsa. When Jenkins tries to clone the repo via ssh, it will try to use that key.

In some setups, you cannot run Jenkins as an own user account, and possibly also cannot use the default ssh key location ~/.ssh/id_rsa. In such cases, you can create a key in a different location, e.g. ~/.ssh/deploy_key, and configure ssh to use that with an entry in ~/.ssh/config:

Host github-deploy-myproject
    HostName       github.com
    User           git
    IdentityFile   ~/.ssh/deploy_key
    IdentitiesOnly yes

Because all you authenticate to all Github repositories using git@github.com and you don't want the above key to be used for all your connections to Github, we created a host alias github-deploy-myproject. Your clone URL now becomes

git clone github-deploy-myproject:myuser/myproject

and that is also what you put as repository URL into Jenkins.

(Note that you must not put ssh:// in front in order for this to work.)

sschuberth
  • 28,386
  • 6
  • 101
  • 146
Mark Longair
  • 446,582
  • 72
  • 411
  • 327
  • Yeah - already managed to solve my problem - I've added ssh key directly to the repo (as deploy key) and not to the account as before. – bx2 Mar 06 '11 at 18:47
  • 4
    Bur how do you created a key for jenkins? – Thiago Diniz Jun 10 '11 at 13:55
  • 5
    The "deploy key" is just any old SSH key. What I did is run `ssh-keygen` as the user Jenkins runs as ("jenkins" on my Ubuntu server). I then added `~jenkins/.ssh/id_rsa.pub` to the deploy keys section of the repository on github. – Adam Monsen Jul 06 '11 at 21:13
  • This answer doesn't help much. Looking for the process of making the keys particularly. I think that you have to login as whatever system account tomcat/jenkins is running as and gen the keys then scoop them up out of /var/empty. – Rob Sep 01 '12 at 01:57
  • 10
    on some installs you would need to out this not into `~` directory. But into `/var/lib/jenkins/.ssh/` for the default jenkins user to use those keys! – garmoncheg Jan 04 '13 at 10:08
  • 7
    To follow up on the comment by @garmoncheg, note that `/var/lib/jenkins` *is* the home directory (`~`) for the `jenkins` user. – David Harkness Aug 13 '13 at 23:40
  • Any clue how to use this method when updating submodules for `myuser/myproject` ?? opened up a question at http://stackoverflow.com/questions/25535632/loading-private-repository-submodules-in-jenkins-project – Erik Aug 27 '14 at 22:21
  • 1
    Does anyone know how to get deploy hooks working with this? I'm seeing errors akin to `Could not match github-deploy-myproject:myuser/myproject` in the hook log. I've entered that as my Repo URL and builds do work so it can access GitHub. It's just the post from GitHub that fails to trigger the build. – pogo Sep 08 '14 at 22:08
  • To be clear, this solution does not currently work with Github Post-hooks, so triggering jobs does not work. - https://issues.jenkins-ci.org/browse/JENKINS-18298 – thebringking Dec 01 '14 at 19:38
  • @pogo I found that triggering builds using the github plugin seemed to require that the Repository URL matched the SSH clone URL, as the plugin seems to reconstruct that URL from the post-hook and trigger builds that have a matching repository URL. I have multiple projects, so ended up creating a dummy user with a single SSH key and access to all the private repos, rather than a deployment key, but I suspect changing `Host github-deploy-myproject` to `Host github.com` and using `git@github.com:myuser/myproject` for repository URL may be what you need (based on my setup, YMMV of course) – chrisbunney Apr 08 '15 at 11:24
37

One thing that got this working for me is to make sure that github.com is in ~jenkins/.ssh/known_hosts.

Nilesh
  • 20,521
  • 16
  • 92
  • 148
Edward Samson
  • 2,395
  • 2
  • 26
  • 39
  • This fixed the issue I was having where after setting up a key pair, a git push was failing – chrisbunney Nov 18 '11 at 16:12
  • In my case the easiest way to do this is to do 'sudo su jenkins' as it isn't possible to log in as the jenkins user properly. Once you have the jenkins identity, you can do a manual ssh login to github/bitbucket and accept the remote host key on behalf of the jenkins user. – LOAS Jun 27 '13 at 12:50
  • But what if you making Jenkins Initialization part of your development environment 'bootstrap'. The 'manual' aspect of this doesn't work – TheJediCowboy Jul 16 '15 at 18:03
13

If you need Jenkins to access more then 1 project you will need to:
1. add public key to one github user account
2. add this user as Owner (to access all projects) or as a Collaborator in every project.

Many public keys for one system user will not work because GitHub will find first matched deploy key and will send back error like "ERROR: Permission to user/repo2 denied to user/repo1"

http://help.github.com/ssh-issues/

Sergii Mostovyi
  • 1,361
  • 1
  • 15
  • 19
  • 2
    The answer about using a deploy key works great if you just have a single repository. But when you want a CI server to build projects across multiple repos you're immediately in the position of managing several sets of keys (one pair per repo) and it becomes much easier to take the approach listed in this answer. – cclark Feb 01 '12 at 21:40
  • This guy's guide explains how to set it via different deploy keys using ~/.ssh/config : https://gist.github.com/victorborda/2871029 – Jorge Orpinel Pérez Sep 18 '14 at 18:52
  • @JorgeOrpinel, I believe the approach in the link can prevent github webhooks from triggering builds using the Github plugin. I found a dummy user with a single key and access to all of the repos worked better when I also wanted builds to be triggered by a webhook, because I needed the Repository URL in the build config to match the github clone URL, see my [other comment](http://stackoverflow.com/questions/5212304/authenticate-jenkins-ci-for-github-private-repository#comment47182184_5212358) – chrisbunney Apr 08 '15 at 11:28
7

Jenkins creates a user Jenkins on the system. The ssh key must be generated for the Jenkins user. Here are the steps:

sudo su jenkins -s /bin/bash
cd ~
mkdir .ssh // may already exist
cd .ssh
ssh-keygen

Now you can create a Jenkins credential using the SSH key On Jenkins dashboard Add Credentials

select this option

Private Key: From the Jenkins master ~/.ssh

Ray
  • 16,025
  • 5
  • 31
  • 51
1

I had a similar problem with gitlab. It turns out I had restricted the users that are allowed to login via ssh. This won't affect github users, but in case people end up here for gitlab (and the like) issues, ensure you add git to the AllowUsers setting in /etc/ssh/sshd_config:

# Authentication:
LoginGraceTime 120
PermitRootLogin no
StrictModes yes
AllowUsers batman git
Jon
  • 391
  • 4
  • 8
1

Another option is to use GitHub personal access tokens:

  • Go to https://github.com/settings/tokens/new
  • Add repo scope
  • In Jenkins, add a GitHub source
  • Use Repository HTTPS URL
  • Add the HTTPS URL of the git repo (not the SSH one, eg. https://github.com/my-username/my-project.git)
  • Add credential
    • Kind: Username with Password
    • Username: the GitHub username
    • Password: the personal access token you created on GitHub
    • ID: something like github-token-for-my-username

I tested this on Jenkins ver. 2.222.1 and Jenkins GitHub plugin 1.29.5 with a private GitHub repo.

salomvary
  • 1,072
  • 9
  • 13
-1

An alternative to the answer from sergey_mo is to create multiple ssh keys on the jenkins server.

(Though as the first commenter to sergey_mo's answer said, this may end up being more painful than managing a single key-pair.)

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
orlanthi
  • 212
  • 2
  • 4