1

Here is my script to monitor log file. I wanted to print value 0 if there is no error found and 1 for error found. That's why i used counter variable to calculate the number of error .

Script

1 #!/bin/bash
      2 set -x
      3 PS4='$LINENO: '
      4
      5 source config.sh
      6 source errorList.sh
      7 counter=0
      8 timeout 10s tail -fn0 $logFile | \
      9
     10 while read line ; do
     11
     12    while read value; do
     13
     14        echo "$line" | grep "$value"
     15
     16         if [ "$line" == "$value" ]
     17         then
     18         echo "Error was found: $value"
     19         counter=$((counter+1))
     20         fi
     21    done <errorList.sh
     22 done
     23
     24 echo "counter value is:$counter"
     25 noError=0
     26 if [ $counter -eq $noError ]
     27 then
     28 echo "0"
     29 else
     30 echo "1"
     31 fi
     32

config.sh

#!/bin/bash
logFile=access.log

errorList.sh

#!bin/bash
error1
error2
error3
error4
error5
error1

debug output:

./newtest.sh
+ PS4='$LINENO: '
5: source config.sh
22: logFile=access.log
6: source errorList.sh
22: error1
errorList.sh: line 2: error1: command not found
33: error2
errorList.sh: line 3: error2: command not found
44: error3
errorList.sh: line 4: error3: command not found
55: error4
errorList.sh: line 5: error4: command not found
66: error5
errorList.sh: line 6: error5: command not found
77: error1
errorList.sh: line 7: error1: command not found
7: counter=0
8: timeout 10s tail -fn0 access.log
10: read line
12: read value
14: grep '#!bin/sh'
14: echo error1
16: '[' error1 == '#!bin/sh' ']'
12: read value
14: grep error1
14: echo error1
error1
16: '[' error1 == error1 ']'
18: echo 'Error was found: error1'
Error was found: error1
19: counter=1
12: read value
14: echo error1
14: grep error2
16: '[' error1 == error2 ']'
12: read value
14: grep error3
14: echo error1
16: '[' error1 == error3 ']'
12: read value
14: echo error1
14: grep error4
16: '[' error1 == error4 ']'
12: read value
14: echo error1
14: grep error5
16: '[' error1 == error5 ']'
12: read value
14: echo error1
14: grep error1
error1
16: '[' error1 == error1 ']'
18: echo 'Error was found: error1'
Error was found: error1
19: counter=2
12: read value
10: read line
24: echo 'counter value is:0'
counter value is:0
25: noError=0
26: '[' 0 -eq 0 ']'
28: echo 0
0

Problem: The debug output shows the counter increment worked properly in a while loop. However, the counter was somehow “reset” to 0 after the loop. The cause of this weird problem is the pipe. When we use a pipe, as in command1 | command2, command2 will be executed in a subshell. The changes that happen in a subshell won’t affect the current shell, even if it’s the same variable.

After searching some solution i have found redirect the process substitution resolve this problem but i couldn't understand how can i apply it using the tail command along?

Thanks in advance.

rakibul sadik
  • 11
  • 2
  • Remove the pipe into the outer `while` loop, and instead at the *end* of that loop (i.e. after the `done`), add a redirect from a process substitution: `... done < <(timeout 10s tail -fn0 $logFile)`. See Kaleb Pederson's answer [here](https://stackoverflow.com/questions/2376031/reading-multiple-lines-in-bash-without-spawning-a-new-subshell/2376059#2376059) and the second part of mweerden's answer [here](https://stackoverflow.com/questions/124167/bash-variable-scope/124349#124349). [BashFAQ #24](http://mywiki.wooledge.org/BashFAQ/024) has more options. – Gordon Davisson Mar 18 '22 at 08:48
  • The answer to your question is you can't. At least with a variable. Pipes create subshells, and subshells can not modify the parent environment. Use process substitution as @GordonDavisson suggests. – dan Mar 18 '22 at 13:05

1 Answers1

0

If you've got Bash 4.2 (released in 2011) or later then you can put

shopt -s lastpipe

before the pipeline and the counter value will persist after the pipeline completes. See how does shopt -s lastpipe affect bash script behavior?.

However, a better approach may be to replace the whole pipeline (and printing of the counter value) with

counter=$(timeout 10s tail -fn0 -- "$logFile" | grep -f errorList.sh -c)
declare -p counter

grep can take a list of patterns to search for from a file, with the -f option. It can also count the number of matches, with the -c option. grep also has options for matching whole lines or whole words, and an option for searching for literal strings instead of regular expressions, which may be relevant to you.

declare -p varname displays the value (and attributes) of the variable named varname in an unambiguous way. For debugging, it's generally better than using echo (or even printf) to show the value of the variable.

A few notes on the code in the question:

  • Shellcheck finds several issues with it, including the fundamental problem that was causing the counter value to be unavailable outside the pipeline. Shellcheck is an invaluable tool for finding problems in Bash (and other shell) code.
  • errorList.sh is just a list of error strings. It's not Bash code. It shouldn't have a shebang line (#!/bin/bash), it shouldn't have a .sh extension (.txt would be more appropriate), and it shouldn't be sourced in the code.
  • The purpose of echo "$line" | grep "$value" is unclear. Maybe it's debugging code. In any case, echo "$var" doesn't work in general. See the accepted, and excellent, answer to Why is printf better than echo?. There's an even better option than replacing echo with printf though: grep -- "$value" <<<"$var". For information about <<< see the Here Strings section in the Bash Reference Manual. The -- option to grep is to prevent a leading hyphen in $value causing it to be treated as a grep option instead of a pattern.
pjh
  • 6,388
  • 2
  • 16
  • 17