Improving speed of checkout from git (from github)

Question

I'm using git and TeamCity v6.0 in a Windows environment.

I have just switched over from server side checkouts to checkout on agents because of IO issues, however...

Initial checkout has become exceeding slow from ~ a minute to >45 minutes. Once the repo is cloned everything is OK. Is there some configuration on the agents I can tweak to improve this? Thanks for any replies in advance.

What I see is the initial git files created on the agent and then there is a very long pause and then I get all my data. I can see from the process explorer that git appears to be doing very little and there is no obvious source of throttling.

score 11 · Accepted Answer · answered Apr 03 '12 at 18:13

11

One thing that can cause a slowdown is java ssh implementation used with agent-side checkout, to turn it off set an agent property teamcity.git.use.native.ssh=true.

Also you can set an agent property teamcity.git.use.local.mirrors=true, in this case TeamCity will create a bare clone of your repository and will update build directory from this bare clone instead of github, i.e. initial clone will be done only once.

answered Apr 03 '12 at 18:13

neverov

1,283
8
11

ill give it a go and let u know how i get on. – James Woolfenden Apr 04 '12 at 07:27
@JamesWoolfenden How did you get on? – Peter Mounce May 29 '12 at 13:10
Nothing helped, unfortunately we are lumbered. – James Woolfenden May 29 '12 at 14:27
@JamesWoolfenden even local mirrors didn't help? This is strange, since with mirrors it is just clone from one folder to another on the same machine. – neverov May 29 '12 at 15:17
Everytime i edited the vcs settings TC recreated the repo. So it became an extremely expensive waste of time to investigate. I saw a 10%-20% improvement but was still taking ~hour to sync a repo. Once i have the repo further checkouts are okish. Not quick. – James Woolfenden May 29 '12 at 15:27
@JamesWoolfenden git on agent use one repository per VCS root url, so TeamCity shouldn't recreate a repo unless you edited the url – neverov Jun 11 '14 at 06:33
@JamesWoolfenden my bad, build checkout directory changes if you edit a VCS root, but if you use local mirrors it results only in clone from a local file system, this shouldn't take an hour. It would help if you can provide teamcity-vcs.log with the slow checkouts. – neverov Jun 11 '14 at 06:38
@neverov that issue is four contracts ago... we migrated to using local git master instead of github. we also had a surfeit of agents. – James Woolfenden Jun 11 '14 at 07:27

score 0 · Answer 2 · answered Apr 03 '12 at 14:07

0

If it's fast on another configuration, your problem is not git or github, but the size of the pipe running into the machine.

If you are using TeamCity in the cloud, you will need to talk to JetBrains about their bandwidth. If locally, talk to your sysadmins.

answered Apr 03 '12 at 14:07

Matt Gibson

14,616
7
47
79

Hi Matt, not using TC in the cloud. Same pipe available on the agents as the server previously. – James Woolfenden Apr 03 '12 at 14:10
Odd. Are they on the same network segment? Possibly firewall is configured differently. – Matt Gibson Apr 03 '12 at 14:13
Very strange. How does ping look from server to repo? Could be that DNS is iffy. – Matt Gibson Apr 03 '12 at 14:20
both are locked down in the same way no . I hoped it was something to do with repo size or suboptimal vcs settings. – James Woolfenden Apr 03 '12 at 14:37
Eventually we moved from github to local git, this helped, not that the problem was github per se, just that it was a remote repo. – James Woolfenden Oct 04 '12 at 14:33

Improving speed of checkout from git (from github)

2 Answers2