2

I'm currently creating a GitHub-action that formats python code while also creating annotations. To achieve this, I pipe the stderr of the black python formatter into the std_in of the reviewdog error parser. While doing so, I, however, want to access the exit code of the black formatter. After researching, I found out that the PIPESTATUS is created exactly for this purpose. I, therefore, currently, use the following code trying to retrieve the exit code of the piped black command:

black --check .
echo "Black non-piped exit code: $?"
black --check . 2>&1 | echo "jan"
PC=(${PIPESTATUS[@]})
echo "Black piped exit code: ${PC[0]}"

>
Black non-piped exit code: 0
Black piped exit code: 1

Surprisingly, the piped access code is always 1 even when black returns an 0 exit code. This behaviour goes away when I remove the 2>&1 redirect:

black --check .
echo "Black non-piped exit code: $?"
black --check . | echo "jan"
PC=(${PIPESTATUS[@]})
echo "Black piped exit code: ${PC[0]}"

>
Black non-piped exit code: 0
Black piped exit code: 0

Therefore, it looks if the std_err redirect changes the exit code found in the PIPESTATUS variable. Unfortunately, I could not yet find why this happens and how I can prevent this from happening. I have two other solutions that allow me to achieve the desired behaviour, but I'm very curious about what is going on. I will therefore be very grateful if somebody can explain to me what is going on. Thanks a lot in advance!

Possible other solutions that work

  • Write black output into a variable: This seems to work and currently is my prefered solution.
  • Write black output to file and then input file input into the error parser: This solution seems a bit overkill for what I'm trying to achieve.

System information

  • black: 19.10b0
  • bash: GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
  • os: Ubuntu 20.04
  • kernel: 5.4.0-58-generic

Related to

Solution summary

As pointed out by @charles-duffy the problem with the exit code is caused by the receiving command not being able to handle the stderr. In my case, this was caused because I allowed the action user to supply unsupported flags to the error parser, which caused the error parser to return the help dialogue. Consequently, the exit code of the sending command was changed to 1.

rickstaa
  • 405
  • 5
  • 23
  • 2
    Piping to `echo` is dubious; `echo` does not read its standard input and terminates quickly, leaving the writing process to die because of SIGPIPE. Simply "don't do it". What are you hoping to achieve by piping to `echo`? There has to be a different way to do it, whatever it is — `cat > /dev/null` might be a better choice. – Jonathan Leffler Dec 28 '20 at 23:34
  • @JonathanLeffler Thanks a lot for your comment. The echo statement is meaningless. I have no idea what I thought when I made the sandbox example :facepalm:. Surprisingly, it let me to the solution of the real problem. Namely, a gap in my understanding of where the exit code of a broken pipe ends up. – rickstaa Dec 29 '20 at 00:17
  • 2
    Note that you can change that exit status -- `set -o pipefail` will make a pipeline's exit status reflect any failure in the pipeline, rather than its default behavior of using only the exit status of the last component. (In the `anything | echo` case, `echo` typically is the first thing to exit -- with status 0 as long as it could write to _its_ stdout successfully -- so an exit status of 0 is almost always expected without `pipefail` set). – Charles Duffy Dec 29 '20 at 00:41
  • @JonathanLeffler Thanks a lot for elaborating on your answer! The gh-action now works again. – rickstaa Dec 29 '20 at 00:47
  • 1
    Thanks, but it was @CharlesDuffy who elaborated, not me. I agree with what was said, though. – Jonathan Leffler Dec 29 '20 at 00:48
  • Ah, You are right. I clearly had a too-long day yesterday. Anyhow, thanks to both of you for your help! – rickstaa Dec 29 '20 at 09:15

1 Answers1

2

This is not surprising at all.

Remember, a pipeline connects stdout (and, with your redirection, stderr) of the thing on the left to stdin of the thing on the right.

If the thing on the right doesn't read its stdin, that means the thing on the left can't write to stdout or stderr. Thus, it's expected to have an error if it attempts to perform such a write. Thus, it's expected to have an exit status reflecting that error.

echo does not read from stdin at all; it simply writes to stdout, then exits. Thus, anything on the left-hand side of a pipe feeding into echo won't be able to write content successfully.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • Ah, that makes total sense! I was not aware of that behaviour. I assumed the broken pipe's exit code to end up in `PIPESTATUS[1]` instead of `PIPESTATUS[0]`. Thanks a lot! I now also understand what is happening when I replace the echo command with the error parser. In this case, the behaviour occurs when the user supplies an unrecognized argument to the error parser. As a result, it returns the help prompt, which can not handle the stderr/stdout input. Consequently, the exit code of the first command is changed to `1`. – rickstaa Dec 29 '20 at 00:05