1

For a game development project we have a git repo which has large binary files. They vary between 10-70MB and ~700MB in total checkout size.

I'm using the following software:

  • Windows 7 64-bit
  • Git [git version 1.9.4.msysgit.2] + open-ssh
  • Tortoise windows client 64-bit

When I try to push (initial push) to github I get the following error (always around progress of ~170MB):

git.exe push -v --progress  "origin" master:master

Pushing to git@github.com:totallymorten/ue4-test.git
Counting objects: 407, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (402/402), done.
select: Not enough memory
fatal: sha1 file '<stdout>' write error: Invalid argument
fatal: The remote end hung up unexpectedly
error: failed to push some refs to 'git@github.com:totallymorten/ue4-test.git'


git did not exit cleanly (exit code 1) (573007 ms @ 12-11-2014 16:50:48)

The funny thing is, that if I try exactly the same on my linux workstation it works!

I've tried all combinations of using Putty plink instead of OpenSSH, and I've also tried https:// for alternative protocol. Nothing has worked on windows so far. Though, I get another error message with https:// and I suspect I can increase the POST buffer size to make it work... but I would really prefer SSH.

I know there are various workarounds like sub-modules and git-annex, and there are other posts like:

git-clone memory allocation error

Repack of Git repository fails

but I've found nothing that really helps my issue.

Any comment appreciated,

Thanks,

Morten.

Community
  • 1
  • 1
Morten Madsen
  • 111
  • 2
  • 6
  • 2
    typically, binaries should be excluded from your SCM because of this exact issue. If you can, you should move them to some shared drive like Copy, Google Drive, Dropbox, etc. – ddavison Nov 12 '14 at 16:31

2 Answers2

0

Git is made for version control so small files with code or text for example.

Static files and especially big files should be excluded from your GIT because exact that problem you have. So make a new folder and exclude that binary files and you have no problems.

Put that folder in your .gitignore.

Take a look at that project. If you need that version control for your large files

https://git-annex.branchable.com

René Höhle
  • 26,716
  • 22
  • 73
  • 82
  • I fully agree with this evalutaion, I'm just wondering why it works on Linux? It must mean the Windows implementation is faulty? – Morten Madsen Nov 13 '14 at 09:19
  • Perhaps its 32bit. Then you have not enough memory / The internal structure. – René Höhle Nov 13 '14 at 09:20
  • That could be. At least I can see the openSSH process is 32 bit. ... hmm maybe I can compile it myself or find a 64-bit version? Good point! – Morten Madsen Nov 13 '14 at 09:24
  • You can try it. But to compile git yourself under windows could be very hard ^^ – René Höhle Nov 13 '14 at 09:32
  • Hmm, looks like [msysgit](https://github.com/msysgit/msysgit/releases/download/Git-1.9.4-preview20140929/msysGit-netinstall-1.9.4-preview20140929.exe) self-compiling version works (git 2.x). This download will automatically compile itself on windows. – Morten Madsen Nov 13 '14 at 12:17
0

Ok, so I finally got it to work:

  • Installing CygWin 64-bit and then using the 64-bit version of git (just choose to install git when you select components) it works flawlessly.
  • A drawback is than you cannot use graphical clients like TortoiseGit with it. Some people have gotten it to work using a wrapper but I'm having difficulties getting everything to work... So I'll stick with the command line client for now.

** EDIT **

Actually you CAN use sourcetree directly with CygWin + git 64-bit. Just point to the git executable in the sourcetree settings like this:

  • git binary: c:\cygwin64\bin
  • external dll: c:\cygwin64\bin

And it works!!

I had some problems with TortoiseGit though, but they were mostly:

  • TortoiseGit detects all files as "modified" when you do a "check for modification". Don't know if this is related to Cygwin.
  • When a file is modified, I cannot do an "Add" operation in TortoiseGit. I can commit it directly with the "Commit to master" menu action though..
Morten Madsen
  • 111
  • 2
  • 6