4

A fun little bash teaser to which I'd love an explanation.

Two loop constructs I would have though are identical, clearly are not. Seems there's some difference in piping vs redirecting when doing a while loop.

Input File

Given this sample file called values.txt with this content:

1
2
3
4
5
6

Piping to while

$ value=0; cat values.txt | while read var; do value=`expr $value + $var`; done
$ echo $value
0

Redirecting to while

$ value=0; while read var; do value=`expr $value + $var`; done < values.txt
$ echo $value
21

To be brief, clearly in the first version each iteration of the while loop executes effectively as () and in the second each iteration iterates as {}

The question is not the difference between () and {}. My question is what causes this difference in behavior for while loops?

Is there a logical reason they should behave differently or was it just a bad choice made early on that couldn't be changed for compatibility reasons? Is it ever possible to pipe to while and get {} behavior?

David Blevins
  • 19,178
  • 3
  • 53
  • 68
  • 3
    In the first version, the while part starts in a subshell (because of the pipe). Therefore, $value does not change. In the second part, the while runs inside the current shell and $value is modified. Add an export in the subshell to see the difference. – Gregor Sep 16 '12 at 06:52
  • Thanks, @Gregor but the difference between shell `{}` and subshell `()` was not the question. The question is why is one a shell and one a subshell. – David Blevins Sep 16 '12 at 07:02
  • 1
    Some material on subshell vs. pipe I've found in the meantime. Doesn't reach for a full answer yet. http://www.linuxprogrammingblog.com/pipe-in-bash-can-be-a-trap http://stackoverflow.com/questions/5760640/left-side-of-pipe-is-the-subshell – Gregor Sep 16 '12 at 07:11
  • 1
    The difference isn't in the `while`; it's not a poor choice made early on - using a pipe starts a subprocess regardless of what you pipe it to, otherwise it wouldn't work. Basically you answered yourself: no you can't do what you want without exporting everything each time, and you don't need to because you have a solution already – moopet Sep 16 '12 at 07:44
  • @moopet that the pipe itself creates the subshell and this ultimately has nothing to do with the `while` loop is the perfect answer. Thank you! – David Blevins Sep 16 '12 at 07:58

1 Answers1

7

This is a known problem and is well explained here: http://mywiki.wooledge.org/BashFAQ/024

To quote the most explanatory part:

Different shells exhibit different behaviors in this situation:

  • BourneShell creates a subshell when the input or output of anything (loops, case etc..) but a simple command is redirected, either by using a pipeline or by a redirection operator ('<', '>').
  • BASH creates a new process only if the loop is part of a pipeline.
  • KornShell creates it only if the loop is part of a pipeline, but not if the loop is the last part of it.
  • POSIX specifies the bash behaviour, but as an extension allows any or all of the parts of the pipeline to run without a subshell (thus permitting the KornShell behaviour, as well).

As for the last question: yes, it is possible in certain shells, and in bash only if you have bash >=4.2, and preceed your code with disabling job control and enabling lastpipe option with the following code: set +m; shopt -s lastpipe

stanwise
  • 2,392
  • 1
  • 15
  • 21