git reset --hard -q produces errors about whitespace

Question

I'm writing a pre-commit hook to run my Python Tests and everything is working wonderfully... until I get to a merge conflict. In this merge conflict, some documentation files are introduced that have trailing whitespace and every time I try to commit, I get these messages:

<stdin>:28: trailing whitespace.
Ticket Link: 
<stdin>:54: trailing whitespace.
Then visit `http://app.dev` 
<stdin>:13528: trailing whitespace.
                            //most be provided for ALL resource updates, and 
<stdin>:13531: trailing whitespace.
    "deleted": false,  //indicates if this action resulted in a resource delete; 

warning: squelched 415 whitespace errors
warning: 420 lines add whitespace errors.
fatal: could not open '.git/MERGE_HEAD' for reading: No such file or directory

And then when I open up my editor to write the commit message, it's totally empty.

My pre-commit script is:

#!/bin/sh

RED='\033[0;31m'
NC='\033[0m'

proper_pop() {
    git reset --hard -q 
    git stash apply -q --index && git stash drop -q
}

exit_and_pop() {
    proper_pop

    if [ "$1" -ne 0 ]; then
        echo "${RED}Your code failed the pre-commit hook! Please examine the output and fix your issues!${NC}"
    fi

    exit $1
}

run_and_bail() {
    bash -c "$1";
    ret=$?;

    if [ "${ret}" -ne 0 ]; then
        exit_and_pop "${ret}"
    fi
}

if git rev-parse --verify HEAD >/dev/null 2>&1
then
    against=HEAD
else
    # Initial commit: diff against an empty tree object
    against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
fi

old_stash=$(git rev-parse -q --verify refs/stash)
git stash -q --keep-index
new_stash=$(git rev-parse -q --verify refs/stash)

if [ "$old_stash" = "$new_stash" ]; then
    echo "pre-commit script: No changes to test. Not running."
    sleep 1  # HACK: Editor may erase message if done too quickly, make the programmer read
    exit 0
fi

# If you want to allow non-ASCII filenames set this variable to true.
allownonascii=$(git config --bool hooks.allownonascii)

# Redirect output to stderr.
exec 1>&2

# Cross platform projects tend to avoid non-ASCII filenames; prevent
# them from being added to the repository. We exploit the fact that the
# printable range starts at the space character and ends with tilde.
if [ "$allownonascii" != "true" ] &&
    # Note that the use of brackets around a tr range is ok here, (it's
    # even required, for portability to Solaris 10's /usr/bin/tr), since
    # the square bracket bytes happen to fall in the designated range.
    test $(git diff --cached --name-only --diff-filter=A -z $against |
      LC_ALL=C tr -d '[ -~]\0' | wc -c) != 0
then
    cat <<\EOF
Error: Attempt to add a non-ASCII file name.
This can cause problems if you want to work with people on other platforms.
To be portable it is advisable to rename the file.
If you know what you are doing you can disable this check using:
  git config hooks.allownonascii true
EOF
    exit_and_pop 1
fi

if ! [ -z "$(which git-pylint-commit-hook)" ]; then
    run_and_bail "git-pylint-commit-hook"
fi

if ! [ -z "$(which pep8)" ]; then
    run_and_bail "python hooks/pep8-hook-check.py"
fi

proper_pop

Through my debugging, I have produced the minimal script down to reproduce this error as:

#!/bin/sh

RED='\033[0;31m'
NC='\033[0m'

proper_pop() {
    git reset --hard -q 
    git stash apply -q --index && git stash drop -q
}

exit_and_pop() {
    proper_pop

    if [ "$1" -ne 0 ]; then
        echo "${RED}Your code failed the pre-commit hook! Please examine the output and fix your issues!${NC}"
    fi

    exit $1
}

run_and_bail() {
    bash -c "$1";
    ret=$?;

    if [ "${ret}" -ne 0 ]; then
        exit_and_pop "${ret}"
    fi
}

if git rev-parse --verify HEAD >/dev/null 2>&1
then
    against=HEAD
else
    # Initial commit: diff against an empty tree object
    against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
fi

old_stash=$(git rev-parse -q --verify refs/stash)
git stash -q --keep-index
new_stash=$(git rev-parse -q --verify refs/stash)

if [ "$old_stash" = "$new_stash" ]; then
    echo "pre-commit script: No changes to test. Not running."
    sleep 1  # HACK: Editor may erase message if done too quickly, make the programmer read
    exit 0
fi

proper_pop

Through this, I have learned that the line: git reset --hard -q in proper_pop is the sole culprit (removing it removes the error). There are no other hooks installed, the git version is: 1.8.1.2, but I've also run this on version 2.5.0 and the same issue happens. Does anyone have any idea what's happening?

I tried piping stdout and stderr to /dev/null for that one command, just to see if it worked, and the errors were still printed...

It's completely a problem I can work around, and it really doesn't cause any issues, but I don't want a less technical co-worker to see this error and for it to throw them off (and I'm not sure if this will be printed every time we're merging in something with trailing whitespaces), so I'd love to know what's happening and how to fix this (or tell git reset to ignore trailing whitespace errors).

EDIT:

Here is a full working repo that demonstrates this problem: https://github.com/hjc1710/so-git-hook-question, merely follow the steps in the README.md and you'll get the error, regardless of whether you're in a merge or not.

My git config (for both my workstation, which is on 1.8.x, and my laptop, which is on 2.5.x) can be found here. Sensitive information has been stripped, but none of it should have been relevant.

Answer 1

As the second footnote in this answer points out, git stash apply merely calls git diff ... | git apply --index. The warnings are coming from these commands.

This happens when you have trailing whitespace, stash the files containing trailing whitespace, and then apply the stash. More accurately it happens any time you try to git apply trailing newlines to a body of code

If you create a git patch with trailing whitespace and try to apply it via cat a.patch | git diff --index, the same error results.

As for solving this issue, it would stand to reason that setting core.whitespace=nowarn in the local, global, or system gitconfig would silence it, but this does not seem to be true for me, but YMMV. (Those settings are not being read, it seems). run git config core.whitespace -blank-at-eol, as pointed out in the comments.

The only solution I see, other than the above, is to find the offending line in the git-stash bash script and add the --whitespace=nowarn argument to git apply, yourself. Which I should mention is a fairly non-standard, non-portable solution. You'd have to have everyone do this, most likely every time git updates, too. But if they bother you to no-end -- that's what you can do.

On the bright side, as you mention, it can be worked around, but it might be worthwhile to open a bug on this on git, itself, if I haven't made a mess of things and it is ignoring even the system gitconfig. I would expect it should honour at least that.

Answer 2

Kudos to you for making this so easy to reproduce!

The problem appears to be that the git command inside the hook is writing the whitespace errors to the current tty instead of stdout or stderr. Then, the hook runner is picking up the errors and then incorrectly claiming that it is stdout.

The fix comes from the answer to one of my previous questions.

#!/bin/sh
git stash > /dev/null
unbuffer sh -c 'git stash apply -q --index  && git stash drop -q ' >/dev/null 2>&1

Edit: If you want the unbuffer program on OSX, you can get it using the instructions here.

brew tap homebrew/dupes/expect
brew install homebrew/dupes/expect