0

I’m totally stumped on a git problem trying to do the initial push from my local repo to github.

In R Studio, I added .git (usethis::use_git()) to my project folder, and then ran usethis::use_github(). It created https://github.com/friendly/6135 but then failed when trying to push.

From the command line, I tried to push directly:

friendly@UIT-HHP-31TRNW1 MINGW64 ~/Dropbox/Documents/6135 (master)
$ git remote -v
origin  git@github.com:friendly/6135.git (fetch)
origin  git@github.com:friendly/6135.git (push)

friendly@UIT-HHP-31TRNW1 MINGW64 ~/Dropbox/Documents/6135 (master)
$ git push -u origin master
Enumerating objects: 1827, done.
Counting objects: 100% (1827/1827), done.
Delta compression using up to 4 threads
Compressing objects: 100% (1820/1820), done.
remote: fatal: pack exceeds maximum allowed size
fatal: sha1 file '<stdout>' write error: Broken pipe KiB/s
error: remote unpack failed: index-pack abnormal exit
To github.com:friendly/6135.git
! [remote rejected] master -> master (failed)
error: failed to push some refs to 'git@github.com:friendly/6135.git'

My local repo is large, ~ 11.7 Gb, and that seems to be why the push failed. I added some folders to .gitignore, but that seems not to have any effect.

Can I try manually uploading the local folders to github until they are done? If so, should I also copy the .git folder?

Edit:

Here are the top 10 largest files/folder in the repo. .git is nearly 4 Gb. I added ./students/ to .gitignore, but it is presumably in my history. I'd prefer not to wipe out .git and start over, but will if that is the only simple way.

friendly@UIT-HHP-31TRNW1 MSYS ~/Dropbox/Documents/6135 (master)
$ du -a . | sort -n -r | head -n 10
9582107 .
3964398 ./.git
3964059 ./.git/objects
2866164 ./students
2169492 ./students/2018
1621472 ./students/2018/presentations
1253347 ./_site
773688  ./lectures
773688  ./_site/lectures
527620  ./students/2018/presentations/OrtacOnder_Presentation.mp4

Edit2:

There have been a number of related posts on this topic, suggesting (confusingly) different solutions. For info & to close this: What finally worked for me was:

  • Identify a few very large files (>100 Mb), no longer needed and remove them.
  • Bite the bullet and rm -rf .git (first saving it elsewhere in case I needed to revert).
  • git init to start over
  • git add a few directories at a time, git commit, git push -u origin master
  • rinse, wash, repeat until all folders were pushed
user101089
  • 3,756
  • 1
  • 26
  • 53
  • Have you already commit large folder before add it to .gitignore ? Maybe try to delete .git folder and re-init git with the .gitignore correctly set – ZecKa Dec 10 '21 at 15:01
  • You will have to trim away large files from your existing commits which means rewriting your repository history. – Lasse V. Karlsen Dec 10 '21 at 15:02
  • Does this answer your question? [How to remove/delete a large file from commit history in the Git repository?](https://stackoverflow.com/questions/2100907/how-to-remove-delete-a-large-file-from-commit-history-in-the-git-repository) – Lasse V. Karlsen Dec 10 '21 at 15:02
  • Also note that GitHub allows, but doesn't really like, exceedingly large repositories - https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github – Lasse V. Karlsen Dec 10 '21 at 15:04
  • I was thinking that even though the error message tells us something different, this might have to do with the fact that you are pushing to `master` while perhaps the main branch now is `main`? I had a similar problem a couple of months back, where the error message didn't really told me the truth about while it failed. – Cyclonecode Dec 10 '21 at 15:17
  • Does this answer your question? [Github remote push pack size exceeded](https://stackoverflow.com/questions/15125862/github-remote-push-pack-size-exceeded) Note, you'll want to add the .gitignore in the initial commit to prevent certain files from ever getting committed. You'll probably need to create smaller commits and push smaller sets of commits (reset back to the past, push, reset forward some, push, repeat, etc). I don't know what the limit is, but apparently you can't do 11.7 GB in a single push, so we know it's smaller than that. – TTT Dec 10 '21 at 15:29
  • Check out the question mentioned by @LasseV.Karlsen as well. In a repo this large it's likely you also have some files that are bigger than 100MB, and you'll also have to strip those out if they aren't already skipped by adding the .gitignore to the initial commit. – TTT Dec 10 '21 at 15:36
  • Re: your edit asking if you can do this without starting over, you can. But you still have to re-write. Currently the best way is probably with a python script called [git-filter-repo](https://github.com/newren/git-filter-repo). (I've used it on a 4 GB repo before and it took just a minute to strip out certain directories.) But if you need to get the .gitignore into an earlier commit, it may be easier to just start over rather than re-writing for all the specific rules you need using `git-filter-repo`. – TTT Dec 10 '21 at 15:39
  • 1
    @Cyclonecode I just noticed that the main branch on github was `main`, while `master` on my local repo. I'll try renaming the local branch first. – user101089 Dec 10 '21 at 18:52

0 Answers0