38

I am running a git gc --aggressive on a very large repo (apx 100 gb). It's been running since two nights ago, and as of a couple hours, it has been stuck on: "Compressing Objects: 99% (76496/76777)"

If I Ctrl-C the process, what are the consequences? Will my repo be unusable? My intuition says no, but I'd like some opinions. Thanks!

imyjimmy
  • 711
  • 2
  • 11
  • 18
  • 4
    Linus says `git gc --aggressive` itself is a bad thing, but data should not be lost. – Dustin May 20 '11 at 03:40
  • Note: with git 2.0, a `git gc --aggressive --depth=xx` can help making that command faster: see [my answer below](http://stackoverflow.com/a/22914119/6309) – VonC Apr 07 '14 at 13:37

3 Answers3

43

git is supposed to be always safe from interruptions like this. If you are worried, though, I suggest Ctrl+Z and then run a git fsck --full to make sure the system is consistent.

There are a number of git-config variables which might help your git-gc go faster. I use the following on one particular large repo, but there are many more options to randomly try (or carefully study, whichever).

git config pack.threads 1
git config pack.deltaCacheSize 1
git config core.packedGitWindowSize 16m
git config core.packedGitLimit 128m
git config pack.windowMemory 512m

These only help if your problem is that you are running out of memory.

Note that Brian J Murray reports for best (git-gc) performance, pack.threads should be set to the number of cores you have. Also note the other answer by VonC which says you can trade gc performance against disk usage by setting gc.aggressiveDepth to a smaller value than the default of 250.

Seth Robertson
  • 30,608
  • 7
  • 64
  • 57
  • thank you! I had a git gc that was getting stuck (overnight!) at 99% I tried these settings and it made it through! – mattorb Jul 11 '12 at 14:03
  • Why Ctrl+Z instead of Ctrl+C? Ctrl+Z seems especially dangerous, since it cannot be intercepted to gracefully shutdown like Ctrl+C can. – JDiMatteo Jan 19 '16 at 22:00
  • @JDiMatteo 1) Ctrl-Z gives you options. If you are consistent you can then kill it, or if not resume or try something else. If you kill it, you can't unkill it if something is bad. 2) Ctrl-Z is catchable. Now it is true that SIGSTOP cannot be caught, but Ctrl-Z generates SIGTSTP which *can* be caught. Not that many people do, and I would be somewhat surprised if git in particular did. – Seth Robertson Jan 20 '16 at 02:40
  • 1
    For best performance, `git config pack.threads` should be set to about the number of CPU cores. – Brian J Murray Mar 19 '18 at 21:06
10

FWIW, I just corrupted a repository by aborting git gc with CTRL+C. git fsck now shows the following errors:

error: HEAD: invalid sha1 pointer [...]
error: refs/heads/master does not point to a valid object!
notice: No default references

And quite a few

dangling commit [...]

I'm not going to investigate on this, but I would like to point out that I'm going to avoid aborting git gc.

Malte Clasen
  • 5,637
  • 1
  • 23
  • 28
5

Note: there is an interesting evolution for git 2.0 (Q2 2014):

"git gc --aggressive" learned "--depth" option and "gc.aggressiveDepth" configuration variable to allow use of a less insane depth than the built-in default value of 250.

This is described in commit 125f814, done by Nguyễn Thái Ngọc Duy (pclouds):

When 1c192f3 (gc --aggressive: make it really aggressive - 2007-12-06) made --depth=250 the default value, it didn't really explain the reason behind, especially the pros and cons of --depth=250.

An old mail from Linus below explains it at length.
Long story short, --depth=250 is a disk saver and a performance killer.
Not everybody agrees on that aggressiveness.
Let the user configure it.

That could help avoiding the "freeze" issue you have when running that command on large repos.

Community
  • 1
  • 1
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • What depth have you found makes sense? I have a relatively small repository I am bringing online with Git and it's choking on 97% of 5400 odd files. – Lawrence Dol Jul 25 '15 at 00:56
  • @LawrenceDol What version of Git are you using? – VonC Jul 25 '15 at 06:36
  • Atlassian SourceTree on Windows with embedded Git 1.9.5. But, actually I got it to go through in the end. – Lawrence Dol Jul 25 '15 at 15:12
  • @LawrenceDol ok. On Windows, you can go as high as 2.4.6: https://github.com/git-for-windows/git/releases – VonC Jul 25 '15 at 15:13