git pre-push hook: run test on each new commit

Question

Context

I want to ensure that each commit I push pass tests.

I want to check this on my (client) side, i.e. before commits are even pushed (so I don't want to rely on CI tools).

Problem

Currently, I have implemented a pre-commit hook that run my tests, so that I cannot even commit a broken state.

However, my test suite takes more than a few seconds to run. It is that much time I need to wait prior to writing my commit message. This makes it impractical to use on a daily basis; both because I frequently commit, and that I sometimes purposefully want to commit a broken state to be squashed later (I know about git commit --no-verify, but that is not the point).

Question

So instead of checking each commit one at a time (at creation), I want to batch-test them before pushing.

How to implement a pre-push hook that run my test suite for each new commit to be pushed?

(For the sake of simplicity, say that passing tests means test/run_tests.sh returns 0.)

[`pre-push hook`](https://git-scm.com/docs/githooks#_pre_push) is passed local and remote IDs. To run tests on every commit run a loop over `git rev-list ..`. In the loop checkout every commit into a temporary directory, run tests and remove the directory. — phd, May 05 '20 at 17:44
@phd: there's a bit of an issue here though: the remote ID may not exist locally. Of course if that's the case, the push will probably be rejected as a non-fast-forward in the first place. — torek, May 05 '20 at 17:45

score 3 · Accepted Answer · edited Jun 14 '21 at 09:50

Thanks to phd's hint (in comments) and a shameless pillage of git's own example, I have drafted the following ./.git/hooks/pre-push hook (that I took care to chmod +x beforehand).

It seems to do the job in vanilla situation, we'll see how it goes over time. Anyway, improvements are welcome!

#!/usr/bin/sh

# An example hook script to verify that each commit that is about to be pushed
# pass the `./run_tests` suite. Called by "git push" after it has checked the
# remote status, but before anything has been pushed.
# If the test suite (and so the script) exits with a non-zero status, nothing
# will be pushed.
#
# In any case, we revert to the pre `$ git push` state.


# Retrieve arguments
remote="$1"
url="$2"

z40=0000000000000000000000000000000000000000 # SHA of a non existing commit


# Save current "git state"
current_branch=$(git rev-parse --abbrev-ref HEAD)

STASH_NAME="pre-push-$(date +%s)"
git stash save -q --keep-index $STASH_NAME


# Do wonders
while read local_ref local_sha remote_ref remote_sha
do
        if [ "$local_sha" = $z40 ]
        then
                # Handle delete
                continue # to the next branch
        elif [ "$remote_sha" = $z40 ]
        then
                # New branch, examine all commits
                range="$local_sha"
        else
                # Update to existing branch, examine new commits
                range="$remote_sha..$local_sha"
        fi

        # Retrieve list of commit in "chronological" order
        commits=$(git rev-list --reverse $range)

        # Loop over each commit
        for commit in $commits
        do
            git checkout $commit

            # Run the tests
            ./test/run_tests.sh

            # Retrieve exit code
            is_test_passed=$?

            # Stop iterating if error
            if [ $is_test_passed -ne 0 ]
            then
                echo -e "Aborting push: Test failed for commit $commit,"\
                  "with following error trace:\n"
                # something like: tail test/run_tests.log
                break 2
            fi
        done
done


# Revert to pre-push state
git checkout $current_branch

STASH_NUM=$(git stash list | grep $STASH_NAME | sed -re 's/stash@\{(.*)\}.*/\1/')
if [ -n "$STASH_NUM" ]
then
    git stash pop -q stash@{$STASH_NUM}
fi
#removed fi

# Return exit code
exit $is_test_passed

Mostly good. I improved a few minor shell and git idioms: use `continue` instead of `exit`, use `elif` and avoid excessive indent, use `git rev-parse` to get the current branch, directly use `$?` instead of `echo $?`. — phd, May 05 '20 at 21:39
Am I reading this right that this would run the tests on *each commit*? Isn't the point of OP's question just that they want to confirm the tests run on the head commit before pushing, so as not to worry about possible broken states in intermediate commits? — coppereyecat, Sep 20 '22 at 22:51
@coppereyecat indeed, the question is about running the tests *“for each new commit to be pushed”* — i.e. not only HEAD (because, in this context, I do want to worry about possible broken states in intermediate commits). Whether checking for this is relevant/a good practice would be another question to ask. — ebosi, Sep 25 '22 at 13:23
I apparently didn't notice when I commented that you ARE OP ;) It seems a little contradictory to run them all at once, once for each commit, though. How do you fix a broken commit halfway through the list without a bunch of gnarly history rewrites? — coppereyecat, Sep 26 '22 at 14:26
@coppereyecat the idea is to `git rebase -i` before pushing, so that only clean history reaches the remote. Whether this qualifies as “gnarly history rewrites” is up to you (-;. In that sense, the solution fits the spec… but, again, other practices (e.g. short-lived branches, with PR, and squashed commit) might be overall better than this approach — but that's out of the scope of this question. — ebosi, Sep 27 '22 at 13:08

score 1 · Answer 2 · answered Apr 23 '21 at 11:49

from ebosi's answer i added some modifications that might interest some of you:

Each time i iterate the loop of commits i recompile the test binary (the right version of the test for the right commit). This is done in the shell function "recompile_test" (note: you might want to double check that the variable "is_test_passed" defined in the scope of this function is still valid in the scope of the whole script itself)
if a test succeeds i display a nice "OK message", plus a reminder of the commit message: echo "`git log -1 --oneline $commit`\n"
if a test fails, i dont break and exit! i display a nice "KO message", plus a reminder of the commit message. and a summary of the trace of the tests just like ebosi did in his answer.
This way at the very end. if the very last commit passed the tests i still can push.
also when done debugging i added a couple of "> /dev/null" here and there
also had to modify the sed instruction, so that it works on mac os x (catalina version 10.15.7, sed is a built-in)


# An example hook script to verify that each commit that is about to be pushed
# pass the `./run_tests` suite. Called by "git push" after it has checked the
# remote status, but before anything has been pushed.
# If the test suite (and so the script) exits with a non-zero status, nothing
# will be pushed.
#
# In any case, we revert to the pre `$ git push` state.


# Retrieve arguments
remote="$1"
url="$2"

z40=0000000000000000000000000000000000000000 # SHA of a non existing commit


# Save current "git state"
current_branch=$(git rev-parse --abbrev-ref HEAD)

STASH_NAME="pre-push-$(date +%s)"
git stash save -q --keep-index $STASH_NAME

recompile_test()
{
   echo "\n\033[32mRecompiling tests suite...\033[0m"
   make test 2>&1 > /dev/null
   # Retrieve exit code
   is_test_passed=$?

   if [ $is_test_passed -ne 0 ]
   then
       echo "\033[31m[\033[mKO\033[31m] Aborting push: Tests failed for commit $commit\n"\
           "\t\033[31mCOULD NOT COMPILE TEST SUITE\n\033[0m"
       break
   fi
}

# Do wonders
while read local_ref local_sha remote_ref remote_sha
do
       if [ "$local_sha" = $z40 ]
       then
               # Handle delete
               continue # to the next branch
       elif [ "$remote_sha" = $z40 ]
       then
               # New branch, examine all commits
               range="$local_sha"
       else
               # Update to existing branch, examine new commits
               range="$remote_sha..$local_sha"
       fi

       # Retrieve list of commit in "chronological" order
       commits=$(git rev-list --reverse $range)

       # Loop over each commit
       for commit in $commits
       do
           git checkout $commit > /dev/null 2>&1

           recompile_test;

           echo "\n\033[32mRunning tests...\033[0m"
           # Run the tests
           ./tester 2>&1 > ./tests_logs/last_test_output.log

           # Retrieve exit code
           is_test_passed=$?

           # Stop iterating if error
           if [ $is_test_passed -ne 0 ]
           then
               echo "\033[31m[\033[mKO\033[31m]\033[0m Aborting push: \033[31mTests failed for commit $commit\n\033[0m"
               echo "commit brief:"
               echo "\033[38;5;160m`git log -1 --oneline $commit`\033[0m\n"
               echo "\t\033[31mERROR TRACE:\n\033[38;5;132m"
               cat ./tests_logs/last_test_output.log
               echo "\033[0m"
           else
               echo "\033[32m[\033[0mOK\033[32m] \033[0mTrying to push: \033[32mTests Passed for commit $commit\n\033[0m"
               echo "commit brief:"
               echo "`git log -1 --oneline $commit`\n"
           fi
       done
done

#added so it doesnt prevent us to checkout... we rm if the file exists
[ -f tests_logs/output.xml ] && rm tests_logs/output.xml
# Revert to pre-push state
git checkout $current_branch 2>&1 > /dev/null

STASH_NUM=$(git stash list | grep $STASH_NAME | sed -e 's/stash@{\(.*\)}.*/\1/')
if [ -n "$STASH_NUM" ]
then
   git stash pop -q stash@{$STASH_NUM}
fi

if [ $is_test_passed -ne 0 ]
then
   echo "\033[31m[\033[0mKO\033[31m] \033[0mPUSH ABORTED (failed on last commit)\n"
else
   echo "\033[32m[\033[0mOK\033[32m] \033[0mPUSH POSSIBLE (final commit successful)\n"
fi
# Return exit code
exit $is_test_passed```

git pre-push hook: run test on each new commit

Context

Problem

Question

2 Answers2

Linked