Git - How to remove a large file from commit history so I can push repository?

Question

I've accidentally committed a directory that weighs around 100MB to my repository a few months back.

Fast forward to today, I'm trying to migrate that repo to Github. It won't let me because of a size limit.

I've removed that directory using the following post: Completely remove file from all Git repository commit history

It seemed to have worked however the entire project still weighs 100M. I've run du . and got the following line: 98M ./.git/objects.

I'm not sure how to fix this as I've tried a few solutions and none of them seemed to work.

What should I do now?

Edit: I managed to fix it using the thread I linked above. Specifically Darren's answer.

simular: http://stackoverflow.com/questions/25389155/trying-to-migrate-to-github-from-bitbucket-getting-file-size-error — Alexan, Aug 28 '14 at 14:07
I don't have any branch other than `master`. Either way I've fixed it and added the solution to my question. — matanc1, Aug 28 '14 at 18:21

score 2 · Answer 1 · edited May 23 '17 at 11:57

I managed to fix it using the thread I linked above. Specifically I did something similar to Darren's answer. I'll put it here for your convenience:

This is the best way:
http://github.com/guides/completely-remove-a-file-from-all-revisions

Just be sure to backup the copies of the files first.

EDIT

The edit by [Neon][1] got unfortunately rejected during review. See Neons post below, it might contain useful information!

[1]: https://stackoverflow.com/users/309261/neon

E.g. to remove all *.gz files accidentally committed into git repository:
$ du -sh .git ==> e.g. 100M
$ git filter-branch --index-filter 'git rm --cached --ignore-unmatch *.gz' HEAD
$ git push origin master --force
$ rm -rf .git/refs/original/
$ git reflog expire --expire=now --all
$ git gc --prune=now
$ git gc --aggressive --prune=now
That still didn't work for me? (I am currently at git version 1.7.6.1)
$ du -sh .git ==> e.g. 100M
Not sure why, since I only had ONE master branch. Anyways, I finally got my git repo truely cleaned up by pushing into a new empty and bare git repository, e.g.
$ git init --bare /path/to/newcleanrepo.git
$ git push /path/to/newcleanrepo.git master
$ du -sh /path/to/newcleanrepo.git ==> e.g. 5M 
(yes!)

Then I clone that to a new directory and moved over it's .git folder into this one. e.g.
$ mv .git ../large_dot_git
$ git clone /path/to/newcleanrepo.git ../tmpdir
$ mv ../tmpdir/.git .
$ du -sh .git ==> e.g. 5M 
(yeah! finally cleaned up!)

After verifying that all is well, then you can delete the ../large_dot_git and ../tmpdir directories (maybe in a couple weeks or month from now, just in case...)

In short: I filtered the branch, created a new bare repo, pushed the master to it, cloned it into a new directory and replaced my project's git directory with the git directory from the clone.

score 1 · Answer 2 · answered Aug 28 '14 at 18:23

You need to expire entries in the reflog otherwise it will hold references to those old blobs, then garbage collect it.

git reflog expire --expire=now --all
git gc --prune=now --aggressive

If that doesn't work try the BFG tool to clean your repo. Do as they say, make a copy of the repository first.

Git - How to remove a large file from commit history so I can push repository?

2 Answers2