0

I want to do something a little more complicated than un-submodule a git submodule

I have a top-level git repo that contains submodule repos.

I want to perform a filter-branch history rewrite on the top-level repo, and essentially incorporate the files from the corresponding submodule version at each commit in the top-level history.

This can also be described as the reverse of Convert a git folder to a submodule retrospectively?

For example,

Submodule repo has history 

A--B--C--D

Top repo has history 

X--Y--Z
where
X[points to A]--Y[points to B]--Z[points to D]

The naive solution described in the other question would result in:

A--B--C--D
          \
           > M
          / 
   X--Y--Z

I would like instead:

X'--Y'--Z'
where:
X' is X plus the files from A
Y' is Y plus the files from B
Z' is Z plus the files from D'

Addressing comment from user torek: Note that commit C is lost, and commit D' contains all the changes from commits C and D. This is not by choice, but a simplification. I would also be just as happy with a commit C' (the changes from C) just showing up between Y' and Z', unfortunately that introduces other edge cases that are not worth solving for.

What have I tried so far?

Spent some time trying to articulate what I want to do and researching existing SO questions.

Is this an XY problem?

No, it's not. This is not part of some larger problem that I'm actually trying to solve.

Alex R
  • 11,364
  • 15
  • 100
  • 180
  • Worth noting: if the superproject has, say, 3 unique gitlinks to the submodule, but the submodule itself has more than 3 commits, you will almost certainly omit all but three of the submodule commits in the rewritten, de-submodule-ized superproject. If you wish to avoid that you have a much harder problem as there's no guarantee that the superproject commit graph and submodule commit graph are conformable. – torek Sep 21 '19 at 18:41
  • Yes, good catch. I updated the question to show a lost commit from the submodule repo. – Alex R Sep 21 '19 at 18:44

1 Answers1

3

A submodule is a (perfectly ordinary) repository, if you want to read stuff out of its commits you can add its object db to your search.

GIT_ALTERNATE_OBJECT_DIRECTORIES=$PWD/.git/modules/mysubmodule/objects \
git filter-branch --index-filter '
        if subcommit=`git rev-parse -q --verify :path/to/submodule`; then
                git update-index --force-remove path/to/submodule
                git read-tree --prefix=path/to/submodule/ $subcommit
        fi
' -- --all
GIT_ALTERNATE_OBJECT_DIRECTORIES=$PWD/.git/modules/mysubmodule/objects \
git repack -ad

should do it.

jthill
  • 55,082
  • 5
  • 77
  • 137
  • The `git rev-parse` seems to be failing when executed inside the filter. There is no error message, just a non-zero exit code :-( – Alex R Sep 21 '19 at 19:19
  • Removing the `-q` option I get... `fatal: Needed a single revision tree filter failed: git rev-parse --verify :***` where `***` is my submodule path. When I change it back to an `index-filter` the error is the same – Alex R Sep 21 '19 at 19:38
  • Fixed its behavior when path/to/submodule doesn't exist, sorry for oops. but where are you getting a tree filter from? this is an index filter. Ah, I see you were just trying it. – jthill Sep 21 '19 at 19:51
  • Yeah sorry I changed the index-filter to a tree-filter while debugging. It's back to an index-filter now. – Alex R Sep 21 '19 at 19:52
  • Failed halfway on `failed to unpack tree object`... I think there was a deleted commit at some point the submodule's history – Alex R Sep 21 '19 at 20:08
  • If there was ever a not-a-commit at that path you'll have to add some more to the filter to handle that, e.g. add ``&& [[ `git cat-file -t $subcommit` = commit ]]`` to the test. – jthill Sep 21 '19 at 20:24
  • I have it skipping the errors now, I just need to figure out how to have these bad commits dropped from the history – Alex R Sep 21 '19 at 20:36