6

I have a command I would to run to generate random string:

var=`< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c8`

When I run this command in interactive bash session I get absolutely no errors. But when I put this command into script and run it as a script I get Broken pipe error indicated by tr. I've read several related topics but still has no answer why script and interactive behavior is different and is there a way to control it with shell options or with something else?

Edit I:

In regards to comments given I found that indicating broken pipe errors can be controlled via:

 trap - SIGPIPE # to ignore errors

and

 trap "" SIGPIPE # to display errors

Edit II:

Well, I've provided incorrect information about reproduction conditions. Finally it seems that problem caused with the python wrapper that called the script with the os.system():

 python -c "import os; os.system('sh -c \"< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c8\"')"

Line given produces broken pipe errors independently of used OS.

Edit III:

This topic has been discussed here: https://mail.python.org/pipermail/python-dev/2005-September/056341.html

reddot
  • 764
  • 7
  • 15
  • 1
    Sounds like the issue being discussed [here](https://github.com/koalaman/shellcheck/issues/494) and in the referenced issue. (shellcheck isn't involved here it just happens to be a discussion of this exact problem and discussion about having shellcheck, which is a great tool, warning about exactly this sort of thing.) – Etan Reisner Oct 08 '15 at 16:07
  • Works perfectly under Cygwin/bash-4.1.10(4)... (Set up your line plus an `echo $var` in a script, nothing more except `#!/bin/sh`.) – Amessihel Oct 08 '15 at 16:09
  • 1
    Are you trapping SIGPIPE in the script? – Petr Skocik Oct 08 '15 at 16:15
  • `trap - pipe` == reset sigpipe to its default disposition (=kill target); `trap '' pipe` == set the disposition of sigpipe to `SIG_IGN` – Petr Skocik Oct 08 '15 at 17:06
  • `tr -dc _A-Z-a-z-0-9` should apparently be `tr -dc _A-Za-z0-9`. I get "invalid byte sequence" unless I prefix the repro command line with `LC_ALL=C` (the bytes from `urandom` are usually not valid UTF-8, which trips `tr` and generally most character-processing utilities in UTF-8 locales). – tripleee Apr 10 '18 at 04:27
  • Also for the record, the repro command line simply hangs for me on Mac OS High Sierra (10.13) – tripleee Apr 10 '18 at 04:28

3 Answers3

7

If one of the parent processes traps sigpipe, then the pipeline will inherit the ignore signal disposition, which will cause this problem you're experiencing.

This can be (safely) reproduced with:

( trap '' pipe; var=`< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c8` )

Normally, the head -c8 command will be done pretty soon at which point its stdin is closed. Since it's stdin is a pipe connected to the stdout of tr, it now no longer makes sense for tr to write to its stdout. Once it tries to, the system will kill it with SIGPIPE. Unless tr ignores this signal or has inherited the ignore (SIG_IGN) disposition for this signal from its parent. Then a write to tr's broken stdout will simply cause a regular error and set errno to EPIPE at which point tr will most likely stringify and outputs this error to its stderr and exit.

rel
  • 764
  • 5
  • 18
Petr Skocik
  • 58,047
  • 6
  • 95
  • 142
1

This answer provides a good summary of the problem with piping from Python to head, and shows some workarounds.

https://stackoverflow.com/a/30091579/456550

Community
  • 1
  • 1
Christian Long
  • 10,385
  • 6
  • 60
  • 58
1

The problem seems to be that head reads the specified (or default) number of lines from the input stream, prints them, and then quits. So an upstream program in a pipe that is still writing finds the output stream closed. In my opinion, this is a limitation in the design of head itself. You can instead use sed, which reads the whole stream: sed -n "1,10p"is equivalent to head -n10.

Russ Jones
  • 21
  • 4
  • It's particularly subtle because you don't get SIGPIPE error if the input stream is truncated but too small to raise the error (I guess smaller than some buffer). On my system I get exit codes "0 0" with `seq 1 1000 | head && echo ${PIPESTATUS[@]}` but I get codes "141 0" with `seq 1 10000 | head && echo ${PIPESTATUS[@]}` even if in both case the output of `seq` is truncated. – dariober Feb 04 '22 at 17:45