Github - File XYZ is 126.80 MB; this exceeds GitHub's file size limit of 100.00 MB

Question

I have a local repository that a large file was accidentally added to. Now im almost 100 commits behind. When I try to push to GitHub it gives me an error.

I need help doing whatever I need to do to be able to push the rest of this repository.

I do not need this file and can delete it permanently

(Side Note: This is a directory with IPython nodes & code. I have a script auto pushing it which is why I'm almost 100 commits behind. I just noticed that it wasn't sync'd)

Here's what I've tried without success:

git status

On branch master Your branch is ahead of 'origin/master' by 100 commits.
(use "git push" to publish your local commits)
nothing to commit, working directory clean

git filter-branch --index-filter 'git rm --cached --ignore-unmatch "Education/Coursera/Exploratory Data Analysis/Week1/household_power_consumption.txt"' --tag-name-filter cat -- --all

Rewrite d381c7d5037a6a26abb2b5cef06e57d8b86a398b (95/189)rm 'Education/Coursera/Exploratory Data Analysis/Week1/household_power_consumption.txt'
..
Rewrite f639b57714a5d57ff37b9d4a55c1c69fc0b514a8 (176/189)rm 'Education/Coursera/Exploratory Data Analysis/Week1/household_power_consumption.txt'
Rewrite 587c8b65f19315ebeb6627a75bd703a5dbdec208 (189/189)

Ref 'refs/heads/master' was rewritten
WARNING: Ref 'refs/remotes/origin/master' is unchanged

git rm "Education/Coursera/Exploratory Data Analysis/Week1/household_power_consumption.txt"

fatal: pathspec 'Education/Coursera/Exploratory Data Analysis/Week1/household_power_consumption.txt' did not match any files

git push

warning: push.default is unset; its implicit value is changing in Git 2.0 from 'matching' to 'simple'. To squelch this message and maintain the current behavior after the default changes, use:

git config --global push.default matching

To squelch this message and adopt the new behavior now, use:

git config --global push.default simple

When push.default is set to 'matching', git will push local branches to the remote branches that already exist with the same name.

In Git 2.0, Git will default to the more conservative 'simple' behavior, which only pushes the current branch to the corresponding remote branch that 'git pull' uses to update the current branch.

See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode 'current' instead of 'simple' if you sometimes use older versions of Git)

Counting objects: 841, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (578/578), done.
Writing objects: 100% (835/835), 31.79 MiB | 3.14 MiB/s, done.
Total 835 (delta 196), reused 0 (delta 0)

remote: warning: File .git-rewrite/t/Education/Coursera/Exploratory Data Analysis/Week1/household_power_consumption.txt is 74.92 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: warning: File .git-rewrite/t/Education/Coursera/Exploratory Data Analysis/Week1/household_power_consumption.txt is 69.38 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB

remote: error: GH001: Large files detected.
remote: error: Trace: f013f5e75b4f35d07de5d2d9ef1116bd
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File .git-rewrite/t/Education/Coursera/Exploratory Data Analysis/Week1/household_power_consumption.txt is 126.80 MB; this exceeds GitHub's file size limit of 100.00 MB

To git@github.com:rbohac/IPython-Notebooks.git ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@github.com:rbohac/IPython-Notebooks.git'

Answer 1

Ensure that your latest commit is correct because BFG will assume this and use your latest commit to maintain the status of your repo while it rewrites and cleans the Git history. BFG is similar to 'git-filter-branch' but is a utility written for a more specific purpose whereas the git utility is broader with more functionality at the sacrifice of operating less efficiently. Read the documentation on BFG for more specifics as I got all of this information from that source.

brew install bfg
bfg --strip-blobs-bigger-than 50M
git rm --cached XYZ
git commit --amend -CHEAD
git push

BFG will update your commits and all branches and tags so they are clean, but it doesn't physically delete anything. Examine the repo to make sure everything is correct and then clear cache and run 'git gc' to strip out the unwanted data which Git will now recognize because it has been removed from history.

Further:

• https://help.github.com/articles/removing-files-from-a-repository-s-history/
• https://rtyley.github.io/bfg-repo-cleaner/#usage

Answer 2

I had trouble using bfg after installing with brew install bfg even though I usually have great success with homebrew.

This is what worked for me:

• Download the latest bfg from https://rtyley.github.io/bfg-repo-cleaner/#download

• Execute bfg like this, then enter the rest of the commands @davidcondrey recommends

  cd /my/repo
  java -jar ~/Downloads/bfg-1.12.5.jar --strip-blobs-bigger-than 99M
  git rm --cached XYZ
  git commit --amend -CHEAD
  git push 
  

Answer 3

Another solution without using bfg, I have used this script remove-big-file.sh developed by Bruno Grande.

Please read the script before executing, and understand each step

Brief explanation: it contemplates a local rebase, edit the commit where you included the large file (remove it from git not physically) and amend the commit and finally push (this would include any other commit done after you tried to fix it).

I had the same issue, my approach was the following: I deleted the file from git, and also updated my .gitignore and committed those changes and still faced the same error. After doing each step of the script (manually), it was solved, be aware that the push will include all the commits you've done locally.