12

I only recently added a .gitignore file to ignore some specific data files in a subdirectory of mine. I'd really like to go back in time and remove them from previous commits/history as well.

Is this easily accomplished? Where/how should I start?

anonymous coward
  • 12,594
  • 13
  • 55
  • 97
  • 1
    possible duplicate of [How to permanently delete a file stored in GIT?](http://stackoverflow.com/questions/2004024/how-to-permanently-delete-a-file-stored-in-git) – manojlds Jul 22 '11 at 19:56
  • This is not an exact duplicate; it has a unique solution. See my answer. – cdhowie Jul 22 '11 at 20:01

5 Answers5

11

This will be a two-step procedure:

Introduce the .gitignore file

For this to work, the early commits are going to have to have your .gitignore file. So, you need to:

  1. Check out the root commit: git checkout -b temp $(git rev-list HEAD | tail -1)
  2. Add the .gitignore file.
  3. git commit --amend
  4. Rebase all of your existing branches onto "temp". This may or may not work seamlessly, so you might need to fiddle with it if any conflicts spring up.
  5. Check out your working branch and delete the temp branch.

Rewrite history

Now you can automatically remove these files from all of the branches by this command:

git filter-branch --tree-filter 'git clean -f -X' -- --all

This will rewrite all of your branches to remove ignored files. It may take a while depending on the size of your repository.

riper
  • 833
  • 1
  • 9
  • 17
cdhowie
  • 158,093
  • 24
  • 286
  • 300
  • 2
    The last command fails to remove the items because .gitignore ignores files already added into the repo are not ignored by git. So the solution is: http://stackoverflow.com/questions/1139762/gitignore-file-not-ignoring – Rick Love Feb 10 '14 at 10:34
  • @RickLove, isn't that the point of running filter-branch to remove the files? – fabspro Jun 19 '14 at 10:50
  • @RickLove , what is the code for doing that within git-filter-branch? – randyrand Jun 03 '18 at 22:16
  • I believe it is: `git filter-branch --tree-filter 'git rm -r --cached . && git add .' -- --all` – randyrand Jun 03 '18 at 22:27
  • The `--filter-branch` step did not work for me as described here, because I had to also remove subdirectories. Instead I had to do `git filter-branch --tree-filter 'git rm --cached -r . >/dev/null; git clean -d -X -f -- --all`. **(a)** `git rm -r --cached .` to ensure that git clean will not skip any files (`-r` for recursing the directory tree). **(b)** `git clean -d -X -f` requires `-d` in order to be applied recursively and delete directories. **(c)** No `git add` is needed, because `git filter-branch` commits the directory tree as-is at the end of the command. – kdb Apr 16 '20 at 10:40
  • From the documentation of [`git filter-branch --tree-filter`](https://git-scm.com/docs/git-filter-branch): *The new tree is then used as-is (new files are auto-added, disappeared files are auto-removed - neither .gitignore files nor any other ignore rules HAVE ANY EFFECT!).* – kdb Apr 16 '20 at 10:42
  • @kdb That's what the `git clean -f -X` tree filter script takes care of. – cdhowie Apr 16 '20 at 21:29
  • @Alew That's about the most useless possible critique I could imagine. It conveys no information about what failed. Please provide the exact steps that you performed and describe what happened instead of what you expected to happen. – cdhowie Dec 30 '20 at 16:18
4

I've just done this and some of the other answers/comments nearly got me there, but I had to make some changes.

I've used filter-branch to introduce my .gitignore file at the start of history, and apply it to all previous commits.

This is the command:

git filter-branch --tree-filter 'git rm --cached -rf . >/dev/null ; cp ~/Desktop/new-gitignore .gitignore ; git add . ; git clean -d -X -f >/dev/null' -- --all

As you can see, it assumes you have a copy of the wanted .gitignore file on your Desktop (~/Desktop/new-gitignore).

I found that only after I've removed and re-added (git rm ..., git add .) all the files would git clean remove all the files that the new .gitignore is telling it to ignore. Notice its not necessary to add the files again after clean, as tree-filter will just commit everything as is.

I've included >/dev/null to reduce the output of the script, but you can safely remove this if you want to see the whole process.

mcfedr
  • 7,845
  • 3
  • 31
  • 27
  • Thanks - worked a treat, even though somebody put in a half-baked .gitignore on one branch somewhere in the middle of history. BTW git now warns against using git-filter-branch and e.g. use git filter-repo instead. I ignored the warning! – Dipstick Feb 20 '23 at 19:39
1

See removing file from every commit section. It is exactly your case.

But be careful, as this feature as every history rewriting one DOES change every commit. So if you have public repository and someone could have based some work on it - it would lead to problems.

Ivan Danilov
  • 14,287
  • 6
  • 48
  • 66
0

Look at something like below using filter-branch

git filter-branch --tree-filter 'rm -f passwords.txt' HEAD

http://progit.org/book/ch6-4.html

manojlds
  • 290,304
  • 63
  • 469
  • 417
0

git filter-branch is a possible solution. Make sure you understand the implications before you utilize it, as the changes are permanent. You won't be easily able to merge your filtered branch with the old one.

git filter-branch --index-filter 'git rm --cached --ignore-unmatch filename' HEAD

Swap filename for the specific file you'd like gone.

SoftArtisans
  • 536
  • 3
  • 5