1

I don't want to push >100MB files to my repos as data connectivity is a constraint for me.

Is there any way, any script that automatically removes >100MB files (irrespective of their file format) from my commits?

Solution needs to be preferably with a warning along with a list of files it is removing from commits

doesn't require me to type long commands (git or otherwise)

simple and easy to use with any new repo

P.S.

I know there is 100MB limit to adding and pushing files and we get error while pushing to the github server.

I am not interested in pushing data through git lfs service.

I have been using data type omissions in .gitignore file. However, often I like to commit *.pkl (python pickle files) that are <100MB

Rohin Kumar
  • 730
  • 2
  • 10
  • 24
  • I imagine that it is possible to write a script that wraps `git commit` and will, before committing, fetch the list of would-be changes (by using `git status` or somethng), check file sizes and unstage the big ones, then commit the rest. – Sergio Tulentsev Jul 24 '17 at 08:26
  • yeah... it must be possible...with a similar algo... it seems elementary as well... I think a seasoned script writer should probably be able to code in a few mins... – Rohin Kumar Jul 24 '17 at 08:31
  • `git gc` might also help, if the 100 MB result from more commits. – Christoph Jul 24 '17 at 08:33
  • I don't know if it is useful if individual file is >100MB... I am surprised nobody did a workaround to this common github issue... – Rohin Kumar Jul 24 '17 at 08:42
  • 1
    @RohinKumar: "I think a seasoned script writer should probably be able to code in a few mins" - maybe. IF he notices this question AND decides that it's not too much trouble AND it's worth answering. Too many conditions. I think your best bet is to go try implement it yourself. Or pay someone to do that. – Sergio Tulentsev Jul 24 '17 at 08:54
  • :) +1 for ur comment! yeah... probably have to take free time out of the project (can't be anytime soon)...hence trying my luck here... finally may have to do it myself if there is no do-gooder willing to help... :) – Rohin Kumar Jul 24 '17 at 09:07
  • Possible duplicate of [How to remove/delete a large file from commit history in Git repository?](https://stackoverflow.com/questions/2100907/how-to-remove-delete-a-large-file-from-commit-history-in-git-repository) – Lasse V. Karlsen Jul 24 '17 at 10:03
  • it is not about removing what is already committed but not committing large files in the first place – Rohin Kumar Jul 25 '17 at 01:53

2 Answers2

3

If this happened to you:

  1. You commit a file larger than 100MB
  2. You attempt to push and get the error
  3. You revert the commit with git revert which creates a new commit where the large file doesn't exist but leaves the large file in the repository's history of commits
  4. You attempt to push and get the error again

Then doing this may solve your problem

git reset --soft HEAD^
git reset

Do this until you encounter the commit having file size>100MB.

Suppose, you commit 3 more times after large file commit(now there are 4 commits pending), then you have to write this code 4 times.

For more details, you can click here

am2505
  • 2,194
  • 15
  • 19
1

This is probably what you are looking for. This is a pre-commit hook that will reject large files. The script is not so complicated so you should be able to adapt it to your own requirements if needs be.

phd
  • 82,685
  • 13
  • 120
  • 165
Aratz
  • 430
  • 5
  • 16
  • `pre-commit` (http://pre-commit.com/) seems to be the solution I was looking for! and `check-added-large-files` is default pre-commit hook now! Thanks! – Rohin Kumar Jul 25 '17 at 01:42