Jenkins time-out while fetching from Github in 10 mins.

Question

I've stuck with this issue while configuring Jenkins for Nightly build. Please note that the repository project "project1" is large and is about 900MB. Please let me know how should I go around this problem.

Started by user anonymous
Building in workspace C:\Users\user1\.jenkins\jobs\Nightly Build\workspace
Fetching changes from the remote Git repository
Fetching upstream changes from git@github.com:MyOrg/projectgroup/project1
ERROR: Timeout after 10 minutes
FATAL: Failed to fetch from git@github.com:MyOrg/projectgroup/project1
hudson.plugins.git.GitException: Failed to fetch from git@github.com:MyOrg/projectgroup/project1
    at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:612)
    at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:836)
    at hudson.plugins.git.GitSCM.checkout(GitSCM.java:861)
    at hudson.model.AbstractProject.checkout(AbstractProject.java:1412)
    at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)
    at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:557)
    at hudson.model.Run.execute(Run.java:1679)
    at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:509)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:230)
Caused by: hudson.plugins.git.GitException: Command "fetch -t git@github.com:MyOrg/projectgroup/project1 +refs/heads/*:refs/remotes/origin/*" returned status code -1:
stdout: 
stderr: 
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:981)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:920)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.fetch(CliGitAPIImpl.java:187)
    at hudson.plugins.git.GitAPI.fetch(GitAPI.java:229)
    at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:610)
    ... 10 more

score 28 · Answer 1 · answered Feb 24 '14 at 13:12

28

I was about to use Workaround #2 but it seems as of git plugin 2.0.3 this can now be configured in the UI, although it is a bit hidden away and is per project.

Go to the configure screen for a project, Source Code Management section, Git, Additional Behaviors, Add, Advanced clone behaviors, Timeout (in minutes) for clone and fetch operation.

I would use workaround #2 as suggested by DevHopeful_2012 if you want a global setting.

answered Feb 24 '14 at 13:12

boinged

533
4
9

1

Yes, good catch. It appears that JENKINS-20387 was resolved. I was meaning to come back and update my answer. But, you already got it covered. :) – kevinmm Mar 25 '14 at 04:35
I'm sure if they really put their minds to it they could make this even more difficult to find. – user3640967 Jun 16 '16 at 10:20
This solution is only for the git plugin. Not, the github plugin. But if you need a longer timeout, it is easy to drop back to the git plugin. – Ben Mathews Nov 28 '16 at 20:02

score 19 · Answer 2 · edited May 23 '17 at 12:25

19

It looks like this is a known issue. See JENKINS-20445, JENKINS-20387, and several other issues, which seem to be popping up.

According to these bug reports, this is only happening in the newer version of the Git plugin, so you could downgrade, or try some workarounds:

Workaround #1 - create a local bare reference clone to reduce the time, as stated here.

If you have only 30 kb/second throughput from Jenkins to your git repository, you should probably consider cloning a bare copy of the repository to central location on the Jenkins server, then perform the clone with the "Advanced clone behaviour" to use a reference repository. That will significantly reduce the amount of data transferred from the git server to Jenkins.

On my Debian Jenkins machine, I do that with:
$ sudo mkdir -p /var/cache/git/mwaite
$ sudo chown mwaite.mwaite /var/cache/git/mwaite
$ cd /var/cache/git/mwaite
$ git clone --bare https://github.com/jenkinsci/jenkins.git
After that bare clone is available on the Jenkins machine, add that advanced behavior to the job and it should perform much better.

Workaround #2 - increase the timeout to allow for the long initial clone process to complete, using the Git.timeOut property:

java -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60 -jar jenkins.war

This SO question is a good example of how to set properties for the Jenkins service. Also, note that this value is in minutes and not seconds. This workaround is courtesy of David.

The latter workaround worked for me. Although, I must admit that I'd prefer to use the reference repo, but I don't believe that it works with a multiconfiguration job, such as mine. And, be forewarned that it was surprisingly painful to set a property for every slave node and then internally document that we have to set this on all nodes, using large Git repos, going forward.

edited May 23 '17 at 12:25

Community

1
1

answered Nov 15 '13 at 04:18

kevinmm

3,136
4
21
24

1

Why would #1 not work with a multi-configuration job? The use of a reference repository is purely an internal git optimization, so if the clone works without a reference repo it should also work with. – Neil Mayhew Jan 22 '14 at 22:03
If you have multiple slave nodes, there needs to be a reference repository on each node, and it needs to be in the same place on each node, so a job can be configured to find it regardless of which node it runs on. – Neil Mayhew Jan 22 '14 at 22:06
@Neil I have yet to play with the reference option of the Git plugin's configuration. So, if I have a bare clone on each slave, in different paths, how would I specify these in the job? I can see how to add a reference repo, in the advanced behaviors, to my hosted git repo. – kevinmm Jan 23 '14 at 01:21
It looks like I can just keep adding paths and they will be ignored if they don't exist on the slave. It would be nice if nodes somehow had the ability to map a hosted repo to a local reference repo. But, I guess it would work, right? – kevinmm Jan 23 '14 at 01:24
I don't think it is feasible to assume that they will all be in the same path, given that my slaves consist of windows and unix platforms. – kevinmm Jan 23 '14 at 01:27
I have separate jobs for separate platforms, so I use /home/jenkins/reference/myrepo.git for Linux jobs and C:\jenkins\reference\myrepo.git for Windows ones. – Neil Mayhew Jan 23 '14 at 07:07
2

You can actually use $VARIABLES in the reference field. So what I did is that I defined GIT_MIRROR on my nodes and then in the reference field set $GIT_MIRROR/blah.git and it works great. – Tobias Hieta Jan 23 '14 at 16:19
@TobiasHieta: Nice! I'll try that. – Neil Mayhew Feb 03 '14 at 19:50

FacePalm · Answer 3 · 2014-03-11T09:02:15.253

WINDOWS ONLY

There is a possibility that your ssh connection is not configured properly. If the ssh connection to your git repository is not estabilished, Jenkins build may appear as if it hung and eventually timeout.

By default, the Jenkins installer sets up Jenkins to run as a service, which runs as the “Local System account”, NOT your user account. Since the “Local System account” does not have SSH keys or known_hosts set up, “git clone” will fail.

You can just copy C:\Program Files (x86)\Git\.ssh to C:\Windows\SysWOW64\config\systemprofile\.ssh (the “Local System account” home)

Also,

IMPORTANT: make sure your ssh keys do NOT have a password! Jenkins will appear to hang when cloning the repository, but really it’s ssh blocking in the background waiting for you to input your password.

Detailed steps in http://computercamp-cdwilson-us.tumblr.com/post/48589650930/jenkins-git-clone-via-ssh-on-windows-7-x64

1. I have git 2.8.4 installed, not finding .ssh in the installation folder. 2. Jenkins 2.1 is running under Local System Account on my PC, with GitHub plugin and works fine. 3. We are only having this issue on our company Jenkins servers, still investigating. — Jirong Hu, Jun 08 '16 at 18:27

score 4 · Answer 4 · answered Mar 25 '15 at 08:08

I just found the solution for jenkins timout #10 min errors. This error was coming because of low Internet connection or may be the size of you project in github is large, so jenkins not able load that project from git server.For resolve the problem we have to edit "Additional Behaviours" in Git in jenkins project configuration and increase "Timeout (in minutes) for clone and fetch operations" to "60" or "120" minutes. So now whenever jenkins clone the git project from git server, its enough time to load/clone a project from git server.

score 2 · Answer 5 · answered May 10 '16 at 08:48

Workaround #2 worked for me, I had to change jenkins.xml file.

On Windows:

Go to C:\Program Files (x86)\Jenkins
open jenkins.xml file
add -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60 inside <arguments> tag

This is how my configuration looks like

Before:

<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war"</arguments>

After:

<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60 -jar "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war"</arguments>

score 2 · Answer 6 · answered May 10 '16 at 08:55

In Ubuntu 14.04 lts

Goto /etc/default/jenkins edit with vi or gedit

update java orgs like below

Before update, it will be like JAVA_ARGS="-Djava.awt.headless=true"

Update like below JAVA_ARGS="-Djava.awt.headless=true -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=30"

and restart jenkins like

sudo /etc/init.d/jenkins start

score 0 · Answer 7 · answered Dec 12 '18 at 22:43

0

On CentOS/RedHat 7.4 Linux with Jenkins-2.138

Edit /etc/sysconfig/jenkins, modify the JENKINS_ARGS (last line) with the timeout setting mentioned by others in their answers:

# Pass arbitrary arguments to Jenkins.
# Full option list: java -jar jenkins.war --help
#
JENKINS_ARGS="-Dorg.jenkinsci.plugins.gitclient.Git.timeOut=25"

answered Dec 12 '18 at 22:43

Ed Randall

6,887
2
50
45

2

JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60" is the right spot – ILikeCoffee Sep 10 '21 at 12:40

Jenkins time-out while fetching from Github in 10 mins.

7 Answers7

Linked