3

I have a big project with 2 million lines (about 4 GB) which I want to upload to GitHub.

However, it seems to take forever to sync it and the process has been interrupted several times.
Also, when I'm pushing the repository, my whole internet is like almost down...(I think it may because it's uploading too many files). My connection is like 40MB.
I'm using GitHub's GUI now.

I'm wondering that is there a better(faster) way I can sync my local repo with the online repo?

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
jackhao
  • 3,457
  • 3
  • 22
  • 43
  • 1
    @MitchWheat or each line is 20 characters long and has been edited 100 times. I've seen some pretty horrible 'single repo for every project in a whole company' that would prob approach this size, I think it's possible (especially with lots of image/lib/ other binary files in the repo too). Not a good idea, but possible. – Tyson Nov 10 '13 at 00:25
  • @MitchWheat I interpreted his question as he has a big git repo locally (or just synced to a local server) and is now trying to push the entire repo to a empty github repo. – Tyson Nov 10 '13 at 00:41
  • @Tyson Hi thanks so much for your help. I didn't fully really understand. Should I separate them into different, small repos? (It's a complete PHP server). – jackhao Nov 10 '13 at 01:36
  • Hi jackhau, sorry, my comments were aimed at Mitch, but he seems to have deleted his comments since, so now I look like I'm talking nonsensically to myself. As for your git repo, if it is a single PHP project, then 1 repo makes sense. But I doubt it should be anywhere near 4GB - have you added log/cache/other non critical directories and files? Regardless of if your specific repo is too big for what is should be, your question still stands as a valid question for any large repo. – Tyson Nov 10 '13 at 03:00
  • @Tyson Hi yeah i was thinking about that. But we did get rid of all the cache/useless assets already. There are a bunch of images included. I will try to get rid of it. But still, it won't reduce the size to < 3GB... – jackhao Nov 10 '13 at 03:25
  • Sorry about the on hold status. The moderators are usually doing a splendid job, except in your case. Instead of improving the wording of your question, they chose to qualify as "off-topic". Of course, your question is 100% *on*-topic for Stack Overflow. – VonC Nov 11 '13 at 07:06

1 Answers1

3

GitHub policy itself is clear:

For best performance, we recommend repositories be kept under 1GB each. This limit is easy to stay within if large files (typically, binaries) are kept out of the repository. If your repository exceeds 1GB, you might receive a polite email from support requesting that you reduce the size of the repository to bring it back down under 1GB.

In addition, we place a strict limit of files exceeding 100 MB in size.

And more generally, I have explained why Git won't perform well with huge file or huge repo.

Having "one" repo for your project is a legitimate constraint.
But you have to split your sources in several repos, that you can put back together through submodules.

That way, you have one (very small) parent repo which reference several (not too big) source repos, all of them on GitHub.

For static (ie which don't change much or ever) sources like pictures, it is best to store them in another referential entirely, and have hooks in place which can, on checkout, get those resources back in your working directory.

Community
  • 1
  • 1
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250