Removing entry with null SHA1, and finding corresponding commit

Question

So, I inherited a repository where one submodule was removed, and I get the dreaded:

warning in tree 6eb01385fa82fdef80719ec4990bec2e0b591d47: nullSha1: contains entries pointing to null sha1

I tried to fix this using this answer. However this assumes that I know which commit included that tree and I cannot find it:

git log --pretty='%H %T' | grep 6eb01385fa82fdef80719ec4990bec2e0b591d47

... doesn't return anything. Furthermore, using the filter-branch command as the OP of the aforementioned question elicits a complain about a commit, but this is a different tree, and listing this tree with ls-tree also show a pair of null SHA1 entries.

So, in summary:

• I seem to have at least two trees with null SHA1 entries
• One is spotted by fsck but doesn't seem to be attached to any commit
• Another one is is attached to a commit but is not seen by fsck

Maybe I can fix the one belonging to a commit using the aforementioned answer, but how about the orphan tree?

Edit:

Thanks to all suggestions here. Having a copy of the repo on a tmpfs on a fast machine makes testing all this a breeze. Eventually figured out part of the problem:

• The commit filter-branch complains about (e884a3b0) contains tree e057f815a
• Tree e057f815a contains just a tree : 6eb01385f
• Tree 6eb01385f is the tree with two null SHA-1s tha fsck complains about

Now I wonder how to apply the official answer since the troublesome tree isn't a direct child of the commit. As I understand it I should fix/replace 6eb01385f and reinsert that in e057f815a, and maybe then regen e057f815a to insert it in commit e884a3b0. So that would be:

• git ls-tree {badtree} | sed -e '/0\{40\}/d' | git mktree to fix the bottom tree
• git ls-tree {parenttree} | sed -e 's/badtree/fixedtree/' | git mktree to make a parent tree pointing to it
• replace that one in the commit as indicated in that other answer

OK, so tried the hard way:

# Create new tree by removing empty SHA1s
git ls-tree 6eb01385fa82fdef80719ec4990bec2e0b591d47 | sed -e '/0\{40\}/d' | git mktree
0eabc1625026f92b2737e763a087f7c4000f0084

# Create new parent tree by replacing bad tree by fixed tree in parent tree
git ls-tree e057f815aec33a48981921289fc7ab25e9ea1a16 | sed -e 's/6eb01385fa82fdef80719ec4990bec2e0b591d47/0eabc1625026f92b2737e763a087f7c4000f0084/' | git mktree
df56fe08e90f1a30e6467ac2bba50a3d771c9de4

# Create new commit by replacing old parent tree by new parent tree
git cat-file commit e884a3b0040b3940d259cd72d82be20d5eb8d7c3 | sed 's/e057f815aec33a48981921289fc7ab25e9ea1a16/df56fe08e90f1a30e6467ac2bba50a3d771c9de4/' | git hash-object -t commit -w --stdin
b41674793c985ba63bc68b095024ebcb2fbf0370

# Replace old commit by new commit
git replace e884a3b0040b3940d259cd72d82be20d5eb8d7c3 b41674793c985ba63bc68b095024ebcb2fbf0370

So far so good. But the old commit and trees are still there. And if I rey to remove them with:

git filter-branch --force --index-filter 'git rm --cached --ignore-unmatch Somedir1 Somedir2' --prune-empty --tag-name-filter cat -- --all

It complains that I should use "-r", so I use:

git filter-branch --force --index-filter 'git rm -r --cached --ignore-unmatch Somedir1 Somedir2' --prune-empty --tag-name-filter cat -- --all

which runs... but then the submodules have been replaced by directories that have the same name at the same place, so the above also drops a lot of useful files. And fsck still finds the bad tree, and in addition it finds many "dangling tags". Is there a way to just remove the two bad trees and the commit?

Answer 1

Brute force will probably do the trick:

which-commits-use-tree () 
{ 
    local REPLY;
    git rev-list --all --reflog | while read; do
            git ls-tree -dr $REPLY | grep -q $1 && echo $REPLY uses $1;
    done
}

That's sort of tolerable as a one-off on medium-ish repos, it scanned the whole of git's history in about five minutes on my little system. If you've got anything substantially larger, you'll need patience or something heavier duty.

git cat-file --batch-check='%(objectname) %(objecttype)' --batch-all-objects --buffer \
| awk '/commit|tree/{print $1}' | git cat-file --batch | your-scanner-here

is about the fastest way I can think of to dump the entire history structure for bulk scanning, that took six seconds on the git history; on the linux repo it took about 2m30, that's reasonably encouraging. I'm probably not going to write the scanner for this, though.