100

I have the following Git repository topology:

A-B-F (master)
   \   D (feature-a)
    \ /
     C (feature)
      \
       E (feature-b)

By rebasing feature branch I expected to rebase the whole subtree (including child branches):

$ git rebase feature master

A-B-F (master)
     \   D (feature-a)
      \ /
       C (feature)
        \
         E (feature-b)

However, this is the actual result:

      C' (feature)
     /
A-B-F (master)
   \   D (feature-a)
    \ /
     C
      \
       E (feature-b)

I know I can easily fix it manually by executing:

$ git rebase --onto feature C feature-a
$ git rebase --onto feature C feature-b

But is there a way to automatically rebase branch including all its children/descendants?

Adam Dymitruk
  • 124,556
  • 26
  • 146
  • 141
Tomasz Nurkiewicz
  • 334,321
  • 69
  • 703
  • 674
  • 3
    See also [how I'd rebase a whole subhistory -- several branches, with some links between them resulting from merge](http://stackoverflow.com/a/9706495/94687). The unpleasant part of that solution is the need to reset the topic branch refs to the new rebased commits afterwards. – imz -- Ivan Zakharyaschev Mar 14 '12 at 17:01
  • thanks for mentioning the --onto option for git rebase - it solved my issue – jackocnr Mar 05 '14 at 02:24
  • 7
    Shouldn't `$ git rebase feature master` be `$ git rebase master feature`? – hbogert Apr 17 '15 at 12:46
  • Possible duplicate of [Git rebase subtree](https://stackoverflow.com/questions/14504029/git-rebase-subtree) – carnicer Aug 24 '17 at 12:21

4 Answers4

45
git branch --format='%(refname:short)' --contains C | \
xargs -n 1 \
git rebase --committer-date-is-author-date --onto F C^
raphinesse
  • 19,068
  • 6
  • 39
  • 48
Adam Dymitruk
  • 124,556
  • 26
  • 146
  • 141
  • That last part is supposed to be " --onto B C^" – Adam Dymitruk Apr 12 '11 at 07:53
  • Great, "--contains" was the parameter I needed. I only had to correct `xargs` by adding `-n 1` to force `git rebase` being executed every time for each branch. I allowed myself to edit your response. Also I used `C` rather than `C^`, I am not sure why you have used the latter form. Nevertheless great thanks, it is exactly what I was looking for. – Tomasz Nurkiewicz Apr 12 '11 at 19:21
  • 3
    Rebase onto requires the parent of the oldest commit to delimit the start - hence C^ – Adam Dymitruk Apr 12 '11 at 21:02
  • 3
    Doesn't the "git branch" command output a star before the current branch, screwing up this script if one of the branches to rebase is currently checked out? – Mark Lodato Dec 20 '12 at 21:38
  • 2
    Isn't git branch a porcelain command? Is there a way to do this that's a bit more future proof? – Chris Pfohl Aug 19 '13 at 17:26
  • I'm sure there is.. you can however do a `| grep -v '\*'` before passing to xargs. However, since rebase is also a porcelain command, branch can be used for this. – Adam Dymitruk Aug 19 '13 at 23:20
  • 7
    Adam: Not sure that is the way to go, you do want to have lines with the *, you just don't want the * itself. Something like | tr -d \* would be better suited. My question though is: Why is this using --onto B? I thought it should be rebased on top of master. Also isn't C^ not the same as B? so we are rebasing from B(excluding?) to each branch containing C on top of ... B. Wouldn't the result be exactly the same as before? – Marenz Jan 27 '14 at 15:57
  • 1
    I think that `--committer-date-is-author-date` is unnecessary in this case. When you rebase work, it can be a good idea to highlight the fact that it was rebased by letting the commit-date be older than the authored-date. –  Apr 05 '14 at 01:36
  • 5
    Should not that be `--onto F` instead of `--onto B`, as all these commits are aready onto B, and we move them onto **F** ? – Ad N Sep 27 '16 at 14:46
  • @user456814 Actually, `--committer-date-is-author-date` is precisely what's needed (and what I was looking for). Without it the commit hashes will change such that branches D and F will not have an ancestor commit C (the diff will be the same but different commit hash), so neither D nor F will contain C which can be verified by running `git branch --contains=C`. – crass Sep 12 '21 at 02:13
11

A couple years ago I wrote something to handle this sort of thing. (Comments for improvement are of course welcome, but don't judge too much - it was a long time ago! I didn't even know Perl yet!)

It's meant for more static situations - you configure it by setting config parameters of the form branch.<branch>.autorebaseparent. It won't touch any branches which don't have that config parameter set. If that's not what you want, you could probably hack it to where you want it without too much trouble. I haven't really used it much in the last year or two, but when I did use it, it always seemed to be quite safe and stable, insofar as that's possible with mass automated rebasing.

So here it is. Use it by saving it into a file called git-auto-rebase in your PATH. It's probably also a good idea to use the dry run (-n) option before you try it for real. It may be a little more detail than you really want, but it will show you what it's going to try to rebase, and onto what. Might save you some grief.

#!/bin/bash

CACHE_DIR=.git/auto-rebase
TODO=$CACHE_DIR/todo
TODO_BACKUP=$CACHE_DIR/todo.backup
COMPLETED=$CACHE_DIR/completed
ORIGINAL_BRANCH=$CACHE_DIR/original_branch
REF_NAMESPACE=refs/pre-auto-rebase

print_help() {
    echo "Usage:  git auto-rebase [opts]"
    echo "Options:"
    echo "    -n   dry run"
    echo "    -c   continue previous auto-rebase"
    echo "    -a   abort previous auto-rebase"
    echo "         (leaves completed rebases intact)"
}

cleanup_autorebase() {
    rm -rf $CACHE_DIR
    if [ -n "$dry_run" ]; then
        # The dry run should do nothing here. It doesn't create refs, and won't
        # run unless auto-rebase is empty. Leave this here to catch programming
        # errors, and for possible future -f option.
        git for-each-ref --format="%(refname)" $REF_NAMESPACE |
        while read ref; do
            echo git update-ref -d $ref
        done
    else
        git for-each-ref --format="%(refname)" $REF_NAMESPACE |
        while read ref; do
            git update-ref -d $ref
        done
    fi
}

# Get the rebase relationships from branch.*.autorebaseparent
get_config_relationships() {
    mkdir -p .git/auto-rebase
    # We cannot simply read the indicated parents and blindly follow their
    # instructions; they must form a directed acyclic graph (like git!) which
    # furthermore has no sources with two sinks (i.e. a branch may not be
    # rebased onto two others).
    # 
    # The awk script checks for cycles and double-parents, then sorts first by
    # depth of hierarchy (how many parents it takes to get to a top-level
    # parent), then by parent name. This means that all rebasing onto a given
    # parent happens in a row - convenient for removal of cached refs.
    IFS=$'\n'
    git config --get-regexp 'branch\..+\.autorebaseparent' | \
    awk '{
        child=$1
        sub("^branch[.]","",child)
        sub("[.]autorebaseparent$","",child)
        if (parent[child] != 0) {
            print "Error: branch "child" has more than one parent specified."
            error=1
            exit 1
        }
        parent[child]=$2
    }
    END {
        if ( error != 0 )
            exit error
        # check for cycles
        for (child in parent) {
            delete cache
            depth=0
            cache[child]=1
            cur=child
            while ( parent[cur] != 0 ) {
                depth++
                cur=parent[cur]
                if ( cache[cur] != 0 ) {
                    print "Error: cycle in branch."child".autorebaseparent hierarchy detected"
                    exit 1
                } else {
                    cache[cur]=1
                }
            }
            depths[child]=depth" "parent[child]" "child
        }
        n=asort(depths, children)
        for (i=1; i<=n; i++) {
            sub(".* ","",children[i])
        }
        for (i=1; i<=n; i++) {
            if (parent[children[i]] != 0)
                print parent[children[i]],children[i]
        }
    }' > $TODO

    # Check for any errors. If the awk script's good, this should really check
    # exit codes.
    if grep -q '^Error:' $TODO; then
        cat $TODO
        rm -rf $CACHE_DIR
        exit 1
    fi

    cp $TODO $TODO_BACKUP
}

# Get relationships from config, or if continuing, verify validity of cache
get_relationships() {
    if [ -n "$continue" ]; then
        if [ ! -d $CACHE_DIR ]; then
            echo "Error: You requested to continue a previous auto-rebase, but"
            echo "$CACHE_DIR does not exist."
            exit 1
        fi
        if [ -f $TODO -a -f $TODO_BACKUP -a -f $ORIGINAL_BRANCH ]; then
            if ! cat $COMPLETED $TODO | diff - $TODO_BACKUP; then
                echo "Error: You requested to continue a previous auto-rebase, but the cache appears"
                echo "to be invalid (completed rebases + todo rebases != planned rebases)."
                echo "You may attempt to manually continue from what is stored in $CACHE_DIR"
                echo "or remove it with \"git auto-rebase -a\""
                exit 1
            fi
        else
            echo "Error: You requested to continue a previous auto-rebase, but some cached files"
            echo "are missing."
            echo "You may attempt to manually continue from what is stored in $CACHE_DIR"
            echo "or remove it with \"git auto-rebase -a\""
            exit 1
        fi
    elif [ -d $CACHE_DIR ]; then
        echo "A previous auto-rebase appears to have been left unfinished."
        echo "Either continue it with \"git auto-rebase -c\" or remove the cache with"
        echo "\"git auto-rebase -a\""
        exit 1
    else
        get_config_relationships
    fi
}

# Verify that desired branches exist, and pre-refs do not.
check_ref_existence() {
    local parent child
    for pair in "${pairs[@]}"; do
        parent="${pair% *}"
        if ! git show-ref -q --verify "refs/heads/$parent" > /dev/null ; then
            if ! git show-ref -q --verify "refs/remotes/$parent" > /dev/null; then
                child="${pair#* }"
                echo "Error: specified parent branch $parent of branch $child does not exist"
                exit 1
            fi
        fi
        if [ -z "$continue" ]; then
            if git show-ref -q --verify "$REF_NAMESPACE/$parent" > /dev/null; then
                echo "Error: ref $REF_NAMESPACE/$parent already exists"
                echo "Most likely a previous git-auto-rebase did not complete; if you have fixed all"
                echo "necessary rebases, you may try again after removing it with:"
                echo
                echo "git update-ref -d $REF_NAMESPACE/$parent"
                echo
                exit 1
            fi
        else
            if ! git show-ref -q --verify "$REF_NAMESPACE/$parent" > /dev/null; then
                echo "Error: You requested to continue a previous auto-rebase, but the required"
                echo "cached ref $REF_NAMESPACE/$parent is missing."
                echo "You may attempt to manually continue from the contents of $CACHE_DIR"
                echo "and whatever refs in refs/$REF_NAMESPACE still exist, or abort the previous"
                echo "auto-rebase with \"git auto-rebase -a\""
                exit 1
            fi
        fi
    done
}

# Create the pre-refs, storing original position of rebased parents
create_pre_refs() {
    local parent prev_parent
    for pair in "${pairs[@]}"; do
        parent="${pair% *}"
        if [ "$prev_parent" != "$parent" ]; then
            if [ -n "$dry_run" ]; then
                echo git update-ref "$REF_NAMESPACE/$parent" "$parent" \"\"
            else
                if ! git update-ref "$REF_NAMESPACE/$parent" "$parent" ""; then
                    echo "Error: cannot create ref $REF_NAMESPACE/$parent"
                    exit 1
                fi
            fi
        fi

        prev_parent="$parent"
    done
}

# Perform the rebases, updating todo/completed as we go
perform_rebases() {
    local prev_parent parent child
    for pair in "${pairs[@]}"; do
        parent="${pair% *}"
        child="${pair#* }"

        # We do this *before* rebasing, assuming most likely any failures will be
        # fixed with rebase --continue, and therefore should not be attempted again
        head -n 1 $TODO >> $COMPLETED
        sed -i '1d' $TODO

        if [ -n "$dry_run" ]; then
            echo git rebase --onto "$parent" "$REF_NAMESPACE/$parent" "$child"
            echo "Successfully rebased $child onto $parent"
        else
            echo git rebase --onto "$parent" "$REF_NAMESPACE/$parent" "$child"
            if ( git merge-ff -q "$child" "$parent" 2> /dev/null && echo "Fast-forwarded $child to $parent." ) || \
                git rebase --onto "$parent" "$REF_NAMESPACE/$parent" "$child"; then
                echo "Successfully rebased $child onto $parent"
            else
                echo "Error rebasing $child onto $parent."
                echo 'You should either fix it (end with git rebase --continue) or abort it, then use'
                echo '"git auto-rebase -c" to continue. You may also use "git auto-rebase -a" to'
                echo 'abort the auto-rebase. Note that this will not undo already-completed rebases.'
                exit 1
            fi
        fi

        prev_parent="$parent"
    done
}

rebase_all_intelligent() {
    if ! git rev-parse --show-git-dir &> /dev/null; then
        echo "Error: git-auto-rebase must be run from inside a git repository"
        exit 1
    fi

    SUBDIRECTORY_OK=1
    . "$(git --exec-path | sed 's/:/\n/' | grep -m 1 git-core)"/git-sh-setup
    cd_to_toplevel


    # Figure out what we need to do (continue, or read from config)
    get_relationships

    # Read the resulting todo list
    OLDIFS="$IFS"
    IFS=$'\n'
    pairs=($(cat $TODO))
    IFS="$OLDIFS"

    # Store the original branch
    if [ -z "$continue" ]; then
        git symbolic-ref HEAD | sed 's@refs/heads/@@' > $ORIGINAL_BRANCH
    fi

    check_ref_existence
    # These three depend on the pairs array
    if [ -z "$continue" ]; then
        create_pre_refs
    fi
    perform_rebases

    echo "Returning to original branch"
    if [ -n "$dry_run" ]; then
        echo git checkout $(cat $ORIGINAL_BRANCH)
    else
        git checkout $(cat $ORIGINAL_BRANCH) > /dev/null
    fi

    if diff -q $COMPLETED $TODO_BACKUP ; then
        if [ "$(wc -l $TODO | cut -d" " -f1)" -eq 0 ]; then
            cleanup_autorebase
            echo "Auto-rebase complete"
        else
            echo "Error: todo-rebases not empty, but completed and planned rebases match."
            echo "This should not be possible, unless you hand-edited a cached file."
            echo "Examine $TODO, $TODO_BACKUP, and $COMPLETED to determine what went wrong."
            exit 1
        fi
    else
        echo "Error: completed rebases don't match planned rebases."
        echo "Examine $TODO_BACKUP and $COMPLETED to determine what went wrong."
        exit 1
    fi
}


while getopts "nca" opt; do
    case $opt in
        n ) dry_run=1;;
        c ) continue=1;;
        a ) abort=1;;
        * )
            echo "git-auto-rebase is too dangerous to run with invalid options; exiting"
            print_help
            exit 1
    esac
done
shift $((OPTIND-1))


case $# in
    0 )
        if [ -n "$abort" ]; then
            cleanup_autorebase
        else
            rebase_all_intelligent
        fi
        ;;

    * )
        print_help
        exit 1
        ;;
esac

One thing that I've found, since I originally addressed this, is that sometimes the answer is that you didn't actually want to rebase at all! There's something to be said for starting topic branches at the right common ancestor in the first place, and not trying to move them forward after that. But that's between you and your workflow.

Cascabel
  • 479,068
  • 72
  • 370
  • 318
  • Upvoted "use merge instead". I spent several hours trying to get many topic and subtopic branches rebased before trying the merge option, and merge really was much easier to perform, even though the new master was heavily divergent from the original master. – davenpcj May 18 '16 at 21:47
  • 3
    It scares me a little that the answer contains: "I didn't even know Perl yet" - especially since the answer isn't written in Perl... :-) – Peter V. Mørch Aug 18 '16 at 12:33
  • @PeterV.Mørch, meaning? – Pacerier May 02 '20 at 11:00
  • At least I read that as if the author of this answer had known that he needs to write a script for this and decided it should be written in Perl. He then tried to write some Perl but *accidentally* ended up with a script that can be executed with bash (+ some embbeded awk) instead, still thinking he has written some code in Perl. – Mikko Rantalainen Apr 28 '21 at 14:27
5

Building up on Adam's answer to address multiple commits on either of the side branches as:

A-B-F (master)
   \
    O   D (feature-a)
     \ /
      C (feature)
       \
        T-E (feature-b)

here is a more stable approach:

[alias]
    # rebases branch with its sub-branches (one level down)
    # useage: git move <upstream> <branch>
    move = "!mv() { git rebase $1 $2; git branch --format='%(refname:short)' --contains $2@{1} | xargs -n 1 git rebase --onto $2 $2@{1}; }; mv"

so that git move master feature results in expected:

A-B-F (master)
     \
      O`   D` (feature-a)
       \ /
        C` (feature)
         \
          T`-E` (feature-b)

Breakdown of how this works:

  • git rebase $1 $2 results in
A-B--------------------F (master)
   \                    \
    O   D (feature-a)    O`
     \ /                  \
      C                    C` (feature)
       \
        T-E (feature-b)

Note that feature is now at C` and not at C

  • let's unpack git branch --format='%(refname:short)' --contains $2@{1} This will return list of branches that contain C as feature previous location and will format output as
feature-a
feature-b

The previous location of feature comes from reflogs $2@{1} that simply means "second parameter (feature branch) previous location".

  • | xargs -n 1 git rebase --onto $2 $2@{1} this bit pipes above mentioned list of branches into separate rebase commands for each and really translates into git rebase --onto feature C feature-a; git rebase --onto feature C feature-b
  • Very interesting approach! Could you please explain how it works? – Eugen Labun May 21 '21 at 12:45
  • You packed so much knowledge into your answer: git aliases, aliases with multiple commands, using `!` to define shell commands in an alias, using shell functions in git aliases to properly handle positional arguments, accessing git reflog via `@{n}` notation, ... I learned a lot. Thank you, Taras! – Eugen Labun May 30 '21 at 22:05
  • This approach is really nice. Just some thoughts about resistance to errors: The reference is searched via @{1} in reflog. I guess this can have some odd side effects if e.g. is already based on . Depending on if was previously rebased this could result in an error "fatal: log for '' only has 1 entries" or possibly branches will be unexpectedly rebased which contain at its previous location. A safer method would be to record the ID of before rebasing it, so in the worst case all branches get rebased to their already current location. – Ragas Feb 20 '23 at 10:07
  • Additionally the first command is terminated with ";". In the error case this will lead to still trying to rebase all the branches that contain , this might lead to unexpected behaviour. It would be better to use "&&" here to make sure execution is stopped if something went wrong. I haven't tried this out yet, so please correct me If I missaw. – Ragas Feb 20 '23 at 10:08
1

With the git-branchless suite of tools, you can directly rebase subtrees:

$ git move -b feature -d master

Disclaimer: I'm the author.

Waleed Khan
  • 11,426
  • 6
  • 39
  • 70