4

When I run Git Extensions on a very large project (5GB and a lot of files) it takes a long time (and spawns like 100 git.exe processes) on showing me the changes, before staging.

.git folder has: 7.5 GB
only my files: ~20GB

I really don't know what to do to make it work...

Totty.js
  • 15,563
  • 31
  • 103
  • 175
  • 3
    git isn't really meant to be used for repositories of that size. – Daniel Hilgarth Feb 26 '13 at 10:43
  • then what should I use. I heard a lot of times that git scales to that sizes, but doesn't seems in my case. – Totty.js Feb 26 '13 at 11:02
  • Can you cite some of those claims? – Daniel Hilgarth Feb 26 '13 at 11:06
  • I don't have any, like I said, I heard somewhere... – Totty.js Feb 26 '13 at 11:19
  • The thing that really kills git in my experience is binary files: is most of that size binaries? Can you reduce it by e.g. pulling in your dependencies with maven or NuGet, moving images out into a different store, something else? – Rup Feb 26 '13 at 11:54
  • git performance is also hurt by having tens of thousands of files: http://stackoverflow.com/questions/3313908/git-is-really-slow-for-100-000-objects-any-fixes, one potential workaround is outlined here: http://stackoverflow.com/questions/9669882/slow-git-operations – Gavin Feb 26 '13 at 12:08
  • You might want to try just using the command line then and see if the performance is more acceptable. GitExtensions may be doing more work than you need. Git is faster on linux though. On Windows it seems to take quite a bit of time to iterate over the working tree. Also a lot of stuff is cached - once you exceed your system RAM it will start swapping and degrade performance significatly. Install 32GB RAM? :) – patthoyts Feb 26 '13 at 12:52
  • @Rup: yes a lot of binary. It's not a software project, it's more like a business management folder with excel, word, pdf and images files.. – Totty.js Feb 26 '13 at 14:02
  • @patthoyts: 32GB of ram on 32bit win xp? don't think so.. This is an old portable pc too. I don't want to use cmd line because it's more difficult to select the files I want to stage... – Totty.js Feb 26 '13 at 14:13
  • Unless you really need the distributed features then I'd use Subversion. AFAIK the problem is Git attempting to do clever cross-file analysis on pushes and that doesn't really work on binaries. Once I switched my folder of PDF eBooks from Git to Subversion everything worked much better. That said, there may be a more appropriate non-VCS solution e.g. a CMS or a dedicated document store system that would work better for you. – Rup Feb 26 '13 at 14:44
  • I would need but I can live without it.. – Totty.js Feb 26 '13 at 15:04

1 Answers1

2

Git's real kryptonite right now is indeed giant repositories (think GB size files)

People will yell at you about "you shouldn't version control binary files". And if the files are unchanging, they're right...

However, whatever the file size, if a file is changing frequently (you "open it", "change some stuff", and "push save" on a regular basis), there are - literally - multiple versions of that file. By definition, you need version control

Not on Windows

If you don't have to deal with Windows, your answer is easy. There's simply no competition: git-annex is what you're looking for; the no Windows thing sure does hurt though

On Windows

For those of us stuck on Windows, this stackexchange thread and this stackoverflow one both have a few good pointers, but no silver bullets :(

My biggest hopes currently are git-annex possibly making it to Windows, or bup becomming less "bleeding edge"

Community
  • 1
  • 1
Bukov
  • 650
  • 8
  • 19