0

I have a very simple bash script to compute the sum of numbers appear at each line of a file (I know there are better ways for doing this but I actually need this sum as an auxiliary information and the script is supposed to something more later on). The script is as follows:

TOTAL=0;
cat $DATAFILE | while read LINE; 
do
      COUNT=`echo $LINE| awk '{print $2;}'`;
      TOTAL=$((TOTAL+COUNT));
done
echo "Total = $TOTAL";

However, I always get the output "Total = 0". Surprisingly enough if I move the last line inside the while loop, I get the correct result. For example if the input file contains

A 5
B 3
C 6

I get the output 

Total = 5
Total = 8
Total = 14

But the current version always outputs 0. It seems the value assigned to the variable TOTAL is somehow lost.

Can anyone help me out with the issue?

Thanks in advance

MikeL
  • 2,369
  • 2
  • 24
  • 38
  • 1
    Because the right-hand side of a pipeline exits when the pipe is done unless you have `shopt -s lastpipe` active. – Charles Duffy Oct 01 '14 at 14:46
  • 2
    [I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?](http://mywiki.wooledge.org/BashFAQ/024) – fedorqui Oct 01 '14 at 14:49

1 Answers1

8

This is BashFAQ #24. Fortunately, you're only hitting it here because of an unnecessary use of cat -- you have no good reason to have a pipeline in use at all.

The right-hand side of a pipeline (like the rest of its contents, being made up of transient subshells) exits when the pipe is done unless you have shopt -s lastpipe active.

Instead, either add the line:

shopt -s lastpipe

or restructure your loop:

while read LINE; 
do
      COUNT=`echo $LINE| awk '{print $2;}'`;
      TOTAL=$((TOTAL+COUNT));
done <"$DATAFILE"

...or, in a case where you really need to pipe from another process, use process substitution:

while read LINE; 
do
      COUNT=`echo $LINE| awk '{print $2;}'`;
      TOTAL=$((TOTAL+COUNT));
done < <(cat "$DATAFILE")

By the way, if your shebang is #!/bin/bash, not #!/bin/sh, this would be better written as follows:

total=0
while read -r _ count do; 
  (( total += count ))
done <"$DATAFILE"

read can split lines on characters in IFS itself -- you don't need to use awk for that.

By convention, variable names should be all-lowercase unless they represent environment variables or shell builtins.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • 4
    The important information here is that each part of a pipeline is executed in a subshell. That means the `while` loop is executed in a different BASH which terminates -> all variables die with it. – Aaron Digulla Oct 01 '14 at 14:49