How do I fix a git subtree after the upstream project force pushed onto master?

Question

I've been experimenting with using git subtree and have run into the following situation.

I used git subtree to add an external project to my repo, I intentionally kept all of the history for the upstream project as I want to be able to refer to the project's history and also contribute back to the upstream project later.

As it turns out, another contributor to the upstream project accidentally pushed a large file into the master branch. To fix this, the upstream project rewrote history and force pushed onto master. When creating my "monorepo", I included this commit and I would also like to remove it.

How can I update my repository to reflect the new history of the subtree?

My first attempt was to use filter-branch to completely remove the subtree and all history.

git filter-branch --index-filter 'git rm -rf --cached --ignore-unmatch upstream-project-dir' --prune-empty HEAD

Once the old version of the subtree was removed, I could re-add the subtree using the new upstream master. However, this didn't work because for some reason the commit history still shows up in the git log output.

Update

I've wrote up the steps to create a minimally reproducible example.

First create an empty git repo.

git init test-monorepo
cd ./test-monorepo

Create an initial commit.

echo hello world > README
git add README
git commit -m 'initial commit'

Now add a subtree for an external project.

git remote add thirdparty git@github.com:teivah/algodeck.git
git fetch thirdparty
git subtree add --prefix algodeck thirdparty master

Make some commits on the monorepo

echo dont panic >> algodeck/README.md
git commit -a -m 'test commit'

Now attempt to use git filter-branch to remove the subtree.

git filter-branch --index-filter 'git rm -rf --cached --ignore-unmatch algodeck' --prune-empty HEAD

Examine git log output, I am expecting to see only my initial commit.
```
git log
```

Have you tried to git gc --prune=now to throw out the old commits? Are there some refs to the old version commits? — Damiano, Nov 18 '19 at 13:09
I didn't try this yet, but wouldn't `git gc --prune=now` only delete commits that don't appear in `git log`? — csnate, Nov 18 '19 at 20:07
using git branch -all (that I suppose you are using to see the "old" commits) should show also the commits unrelated to your current branch. — Damiano, Nov 19 '19 at 06:54
Actually, I was just doing `git log`, no arguments and I still see the old commits. — csnate, Nov 22 '19 at 21:22
Please can you post your git log --pretty --all --graph? Just to understand your situation — Damiano, Nov 26 '19 at 16:06

score 1 · Answer 1 · answered Jan 10 '20 at 14:39

on your repo, cleanup the history of commits for this remote :
```
git fetch upstream
```
if one of your own commits has a commit that includes the large file, rewrite your history so that this large file is no longer referenced
```
# using one or more of the following commands :
git rebase --interactive
git filter-branch
...
```

With these two steps, the big file will not be referenced anymore by any commit in your repo.
It will additionally be deleted from your hard drive at some point in time, when git runs its garbage collector and the expiration delays for dangling blobs has been reached.

If you have an urgent need to delete this big file ASAP from your hard drive :

Manually run

git gc --prune=now

score 0 · Answer 2 · answered Dec 25 '19 at 00:31

you already got the bad-commit in your history and you need to get rid of it before continuing

let's assume you got master last commit diverted and haven't been able to do anything else (I really don't have your branches at sight, so I need to assume something to start with)

you can checkout to the previous commit and push your branch marker 1 step back (or X steps back) which would be harmless in any case and then pull again

e.g.

git checkout master~1
git branch master -f
git checkout master
git pull

git checkout master~1 to checkout master's parent commit, git warns we are off branches
git branch master -f to force current checkout to become master again, i.e. it actually rewinds master branch to its previous commit (or X previous commit), and from here, it doesn't matter whether upstream did a force or not, we can resume normally, or even go back to above step if needed, we can only pull master again, without losing anything from upstream (which for us could be read-only as well, we won't be pushing anything for this)
git checkout master to be on our "rewound" master branch, the same commit we are stepping at, but now being on the branch instead
git pull to pull master again (can be with or without --prune), if upstream diverted, we'll get back on track from here, if not, we'll get the same we had, if we got the same and was not supposed, perhaps we need to go back to the 1st step above and rewind more commits, e.g. git checkout master~5 or whatever (as needed)

@csnate it is possible to checkout previous commits from a subrepo and follow very similar procedure, if you build a MCVE it would be easier to tell you the exact commands to follow https://stackoverflow.com/help/minimal-reproducible-example — arhak, Dec 30 '19 at 13:15
I created a set of steps in the original question that shows the issue. — csnate, Feb 05 '20 at 15:16

How do I fix a git subtree after the upstream project force pushed onto master?

2 Answers2