Rebase git submodule and parent repo

Question

My situation: You have a feature branch based off master, and someone commits to master. Now your history is like this:

A - B - C - D (master)
         \
           E - F - G  (feature)

So you'd like to rebase feature onto master for a clean history, as one does. But consider this: that repo is a submodule of another, and the parent repo references the submodule's commits like so:

A - B - C - D (submodule:master)
         \
           E - F - G  (submodule:feature)
           *    *
           *     *
           X - Y - Z             (parent:feature)
              (asterisks represent references to submodule)

If I rebase the submodule naively, the parent's references to submodule's commits will be invalid. Let's assume that some of the commits in the feature branches are meaningful enough to separate, so squashing them into one commit is out.

Any way to do it and maintain those references? (both 'feature' branches can be freely rewritten).

Answer 1

the parent's references to submodule's commits will be invalid.

Those references only becomes invalid if they are lost by the rebase.

All you need to do is add a old_feature branch where feature is, before rebasing feature.
Don't forget to push the old_feature branch.

Then, once feature is rebased and pushed, you go to the parent repo, make sure its submodule follows the feature branch, and do an update remote:

git submodule update --remote

The commits X and Z can keep their old references to old_feature branch, while a new commit will keep a reference to the rebased feature branch of the submodule.

---

As jthill adds in the comments, reflog is still there locally, for 90 days, if you already rebased without referencing the old state of feature first:

you've got a month to fix the oops in that repo, git branch old-feature feature@{last.monday}

Then push that old-feature branch, in order to make sure E and F keep being referenced in the remote repository whose commits are referenced by X and Z.

Answer 2

I face this situation pretty often in my company, and we found 2 main solutions to this problem. Both of them use the ability to git to keep commits in history in certain situations. By the way, the good practice is to never push submodules refs that are pointing to deletable branches (i.e. feature branches).

I assume that, like me, you dont want to push feature_branch in order to not pollute your remote repository and be robust to inquisitor-mode work mates that clean all old pushed feature branches

The problem is that when you do a rebase, commits are beeing re-played on top of history of master. They are new commits (let's say X', Y' and Z'), with new hash, and branch's head is moved on the top of Z'. So X,Y,Z will stay on a dead branch and will be deleted with time (90 day with default settings). In addition to that, they will not be pushed by default when you will push you rebased branch.

1. Classic, clean way : do a merge instead of a rebase

Do a merge. E,F,G will be linked to the merge commit. In that way, X,Y,Z will remain for ever and unchanged, even if the branch is deleted (only the ref of the branch would be in that case. But at the expense of a un-linear history on master). When you will push parent branch, X,Y,Z will be pushed too.

2. Keep commits in history with tags

One solution to keep X,Y,Z commits in history is to tag Z (before the rebase (tag will survive to it), or after (but you will have to re-found it in the history with reflog), it doesn't matter). If this tag is then pushed, X,Y,Z will not be cleaned. This is pretty dirty and has side effects, but does the job. Of course, you will have to push that tag ! (which is pretty equivalent to push a branch, to be honest. The difference is that you can name it specifically, for example DONT_DELETE_THIS_FOOL ;) )

Here is a small script that you can run to ensure that any submodule dependency will not be lost (by tagging each submodule and push them) (use -h to get help):

#!/bin/bash

check_tags () {
    slength=${#1}
    slength=$((slength+1))

    refmajor=`echo $2`
    refminor=`echo $3`
    refrevision=`echo $4`

    echo "Check for anterior tags in $5..."
    for tag in $(git tag -l -n "$1*.*.*" | cut -d" " -f1 | cut -c "$slength"- ); do
        temptagMajor=`echo $tag | cut -d. -f1`
        temptagMinor=`echo $tag | cut -d. -f2`
        temptagRevision=`echo $tag | cut -d. -f3`

        if [[ temptagMajor -gt tagMajor ]] || [[ temptagMajor -eq tagMajor && temptagMinor -gt tagMinor ]] || [[ temptagMajor -eq tagMajor && temptagMinor -eq tagMinor && temptagRevision -gt tagRevision ]]; then
            tagMajor=$temptagMajor
            tagMinor=$temptagMinor
            tagRevision=$temptagRevision
        fi
    done

    echo "Latest versioning tag found : $1$tagMajor.$tagMinor.$tagRevision (want to apply $1$refmajor.$refminor.$refrevision)"

    if [[ tagMajor -gt refmajor ]]; then
        echo "Cannot tag with $1$refmajor.$refminor.$refrevision : anterior tag with greater major revision number (found $tagMajor, wanted $refmajor). Use -f to ignore."
        return 1
    elif [[ tagMajor -eq refmajor ]] && [[ tagMinor -gt refminor ]]; then
        echo "Cannot tag with $1$refmajor.$refminor.$refrevision : anterior tag with equal major revision number but greater minor revision number (found $tagMinor, wanted $refminor). Use -f to ignore."
        return 1
    elif [[ tagMajor -eq refmajor ]] && [[ tagMinor -eq refminor ]] && [[ tagRevision -gt refrevision ]]; then
        echo "Cannot tag with $1$refmajor.$refminor.$refrevision : anterior tag with equal major and minor revision number, but greater revision number (found $tagRevision, wanted $refrevision). Use -f to ignore."
        return 1
    elif [[ tagMajor -eq refmajor ]] && [[ tagMinor -eq refminor ]] && [[ tagRevision -eq refrevision ]]; then
        echo "Cannot tag with $1$refmajor.$refminor.$refrevision : anterior tag with equal major, minor, and revision number (found $1$tagMajor.$tagMinor.$tagRevision, wanted $1$refmajor.$refminor.$refrevision). Use -f to ignore."      
        return 1
    else
        return 0
    fi
}

function subcheck() {  

  if [[ ! -d .git ]]; then
        echo "not a git repo"
        return 1
    fi;

    if ! [[ `git submodule status` ]]; then
        echo 'no submodule'
        return 1
    fi

    submodules=($(git config --file .gitmodules --get-regexp path | awk '{ print $2 }'))

    currentDirectory=$(pwd)

    for submodule in "${submodules[@]}"
    do
        printf "\n\nEntering '$submodule'\n"
        cd "$currentDirectory/$submodule"

        check_tags $1 $2 $3 $4 $submodule

        if [[ $? -eq 1 ]]; then
            cd "$currentDirectory"
            return 1
        fi

    done

  cd "$currentDirectory"
}

#export -f check_tags

# Check arguments
while getopts v:hfti: option
do
case "${option}"
in
v) VERSION=${OPTARG};;
h) HELP='help';;
f) FORCE='force';;
t) TEST='test';;
i) INDEX=${OPTARG};;
esac
done

printf "Tag release script v0.1\n"

# Help
if [ "$HELP" != "" ]; then 
  echo 'GIT Release Script'
  echo "Options :"
  echo 'Use -v to specify version (mandatory). Ex : "-v 1.0.2"' 
  echo 'Use -t to run unit test of -v inputs' 
  echo 'Use -f to force tagging / skip anterior tag versions check' 
  echo 'Use -i to specify index (optional). Ex : "-v 1.0.2 -i A" for a indA + v1.0.2 double tag.'
  exit 1 
fi

# Tests for bad inputs
if [ "$HELP" != "" ]; then 
    array=( ".2.3" "1..3" "1.2." "A.2.3" "1.A.3" "1.2.A" "1A3.123.123" "123.1D3.123" "123.123.1A3" "nougatine" "1.3" )

    arr=(${array[*]})
    echo "Tested valudes : ${#arr[*]}"
    for ix in ${!arr[*]}
    do
        printf "   %s\n" "${arr[$ix]}"
        . release_script.sh -v ${arr[$ix]}
    done
fi

# Version
if [ "$VERSION" == "" ]; then 
  echo "Argument missing"
  echo "Run -h for help"
  exit 1 
fi

major=`echo $VERSION | cut -d. -f1`
minor=`echo $VERSION | cut -d. -f2`
revision=`echo $VERSION | cut -d. -f3`

if [ -n "$(printf '%s\n' "$major" | sed 's/[0-9]//g')" ] || [ "$major" == "" ]; then 
  echo "Invalid major version argument (was \"$major\")"
  echo "Run -h for help"
  exit 1 
fi

if [ -n "$(printf '%s\n' "$minor" | sed 's/[0-9]//g')" ] || [ "$minor" == "" ] ; then 
  echo "Invalid minor version argument (was \"$minor\")"
  echo "Run -h for help"
  exit 1 
fi

if [ -n "$(printf '%s\n' "$revision" | sed 's/[0-9]//g')" ] || [ "$revision" == "" ]; then 
  echo "Invalid revision version argument (was \"$revision\")"
  echo "Run -h for help"
  exit 1 
fi

# Fetching
git fetch --all
echo "Fetching tags"
git fetch --tag

tagMajor=0
tagMinor=0
tagRevision=0

versionLabel=v$VERSION
url=$(git config --get remote.origin.url)
basename=$(basename "$url" .git)

# Check previous available versions if no -f specified
if [ "$FORCE" == "" ]; then 

    if [ "$INDEX" != "" ]; then
        echo "Check for anterior index tag in $basename..."

        for tag in $(git tag -l -n "ind$INDEX" ); do

            if [[ "ind$INDEX" == $tag ]] ; then
                echo "\"ind$INDEX\" tag already exists in $basename. Use -f to ignore."
                exit 1
            fi
        done

        echo "No conflict found for "ind$INDEX" index tag"
    fi

    check_tags v "$major" "$minor" "$revision" "$basename"

    if [[ $? -eq 1 ]]; then
        #echo "Error found, won't tag"
        exit 1
    fi  

    subcheck ${basename}_v $major $minor $revision

    if [[ $? -eq 1 ]]; then
        exit 1
    fi
fi

# Release tag script
versionLabel=v$VERSION
url=$(git config --get remote.origin.url)
basename=$(basename "$url" .git)
echo "Tagging project $basename (\"$versionLabel\")"
git tag $versionLabel

#TODO : check index
if [ "$INDEX" != "" ]; then
    echo "Tagging index $INDEX"
    git tag "ind$INDEX"
fi
echo "Tagging submodules (\"${basename}_$versionLabel\")"
git submodule foreach "git tag ${basename}_$versionLabel || :"
echo "Pushing project tag"
git push --tags
echo "Pushing submodules tags"
git submodule foreach 'git push --tags || :'

Answer 3

A - B - C - D (submodule:master)
         \
           E - F - G  (submodule:feature)
           *    *
           *     *
           X - Y - Z             (parent:feature)
              (asterisks represent references to submodule)

To rebase a submodule (e.g. feature onto master) and update the parent's gitlinks:

1. Create another branch at the submodule's rebase tip (like 'old_feature' at 'feature').

2. Do the rebase in the submodule.

3. Note that you can figure the mapping between the old submodule commits (E -> G) and the new (E' -> G') and their ids. You'll need these later.

4. You should also know the range of commits to alter in the parent repo (X -> Z), and which specific commits have gitlinks that need to be updated.

5. Do an interactive rebase in parent on said commits. Insert a 'break' command after each that needs to be altered.

6. As git drops to shell each time:

   1. For the current parent commit's old gitlink, checkout in the submodule the appropriate new commit.

   2. In the parent, stage the submodule and git commit --amend. This updates the gitlink.

   3. Continue rebase. If there are conflicts (should be a lot), prefer the ones with gitlinks to submodule's old commits, as these conflicts occur before we update them. (when I did it this was the option "use the remote changes" in local vs remote in git mergetool).

7. Done

---

It's rather involved (and potentially slow if you have a lot) and it's a question of whether you think it's worth it.

The easier way, per @NicolasVoron

1. Classic, clean way : do a merge instead of a rebase

Do a merge. E,F,G will be linked to the merge commit. In that way, X,Y,Z will remain for ever and unchanged, even if the branch is deleted (only the ref of the branch would be in that case. But at the expense of a un-linear history on master). When you will push parent branch, X,Y,Z will be pushed too.