3

I recently installed Git-LFS to manage large files. I've quickly reached the 1 Gb storage limit, however, and now when I try to push commits I'm prompted with:

batch response: This repository is over its data quota. Purchase more data packs to restore access.

and failure to push. So now I can't actually push to the repo.

Purchasing more data packs is not an option, however storing large files locally (i.e. not having them version controlled) is.

So what I would like to do is:

  1. Stop monitoring the files that LFS monitors (currently set to all *.csv in .gitattributes).
  2. Remove those files from git, i.e. so that they don't contribute to any repo size.
  3. Still have those files present locally.
  4. Uninstall Git-LFS.
  5. Disrupt the history as little as possible, ideally so only the removed files are affected.
  6. Now that the repo size should be smaller, get back to being able to push/pull as normal.

I've found bits and pieces of info around where people have exceeded the limit, but nothing that can do the above points.

FWIW I typically use Tortoise Git but of course have Git Shell too.

jhpratt
  • 6,841
  • 16
  • 40
  • 50
conor
  • 1,204
  • 1
  • 18
  • 22
  • Once the files are in history, you'll have to rewrite history to remove them. GitHub counts _all_ data, not just current, towards the 1 GB limit. – jhpratt Mar 20 '18 at 01:51
  • Is there a way to rewrite history such that it omits the csv files but otherwise is the same? – conor Mar 20 '18 at 01:54
  • You could setup your own lfs server, locally or at some place in internet, and migrate there. [Here](https://github.com/git-lfs/git-lfs/wiki/Implementations) is list of implementations – max630 Mar 20 '18 at 04:21
  • https://github.com/git-lfs/git-lfs/issues/641 maybe this can help you. – S.Y. Wang Mar 20 '18 at 18:55
  • "You could setup your own lfs server" I happened to do this before pushing anything to GitHub, but GitHub's push hook blocks your push if you have any LFS pointers in your commits (which is ridiculous), forcing you to upload blobs to GitHub, and once you do that, you exhaust your quota and cannot interact with your repo on GitHub at all. – alexei Apr 27 '21 at 01:34

2 Answers2

2

Is there a way to rewrite history such that it omits the csv files but otherwise is the same?

That is what git filter-branch is for. Again, it will rewrite the history, so a git push --force will be needed.
See also "Git - Remove All of a Certain Type of File from the Index": BFG repo cleaner can be easier/faster to use.

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • This answers my comment but not the original question(s). For example, I can't push right now in the current state, so I'm worried that trying this may just lead to a more complex issue rather than a solution. It also isn't clear to me if this will keep the csv files locally or not, how this will affect LFS, or how it will affect my ability to push/pull. – conor Mar 20 '18 at 06:36
  • For testing, you can apply my answer, and then push to another brand new empty repo: see if the result is ok or not there without complicating your original repo. – VonC Mar 20 '18 at 06:38
  • Just a note: there is a typo, revrite -> rewrite – Yue Lin Ho Mar 21 '18 at 03:21
  • @YueLinHo Thank you. I have fixed the typo. – VonC Mar 21 '18 at 05:30
  • How did you resolve this issue? [Another similar question](https://stackoverflow.com/questions/51051553/github-repository-data-limit-reached) was resolved by contacting GitHub Support. – Lubed Up Slug Aug 06 '19 at 04:19
1

If you exhuasted GitHub's free quota, but you don't care about using GitHub's LFS storage, then you can still push to GitHub commits with pointers to the LFS blobs (without uploading the blobs themselves) by "uninstalling" git lfs before pushing:

$ git push origin master

batch response: This repository is over its data quota.
Account responsible for LFS bandwidth should purchase more 
data packs to restore access.

$ git lfs uninstall
$ git lfs uninstall --local

$ git push origin master
# Push worked!

But, if you get the GH008 error (pre-receive hook declined) when you try to push commits with pointers, then the above trick may or may not help. Sometimes it works sometimes it doesn't, I don't fully understand the pattern:

$ git push origin master
...
remote: error: GH008: Your push referenced at least 9 unknown Git LFS objects:
...
! [remote rejected] master -> master (pre-receive hook declined)
alexei
  • 2,031
  • 1
  • 26
  • 28