0

I am modifying a shell script, which uses Makefile to test code in different test benches, currently the shell script only echo SUCCESS and FAILURE, i want to modify this shell script to search through the output trace strings which are being written into another file at the same location where the Makefile is called upon namely trace.log. If it finds the word "ERROR"(which is a trace string) i want the test to say FAILURE

!/usr/bin/env bash
#set -e

BASEDIR=$(pwd)
REPOROOT=`git rev-parse --show-toplevel`


RED="`tput setaf 1`"
GREEN="`tput setaf 2`"
BLUE="`tput setaf 4`"
NORM="`tput sgr0`"

Test() {
    pushd ${REPOROOT}/rr/Fmda/body > /dev/null
    tests=`find . -type f | grep "test.xml" | sed "s/test\.xml//"`
    for t in $tests; do
      testname=`echo $t | sed "s/\.\///" | sed "s/\/test\///"`
      echo -e "${BLUE}EXECUTING $testname TEST${NORM}"
      pushd $t > /dev/null
      make test_svp
      if [ $? -eq 0 ]; then
        echo -e "${GREEN}SUCCESS IN $testname TEST MOVING ON!${NORM}"
      else
        echo "${RED}FAILURE IN $testname TEST${NORM}"
        exit 1
      fi
      popd > /dev/null
     done
     popd > /dev/null
}

I have no background in shell scripting. I understood what's written here by googling the constructs, but still I didn't understand how for t in $tests works and what sed does.

From what i thought would work for my problem is

error=trace.log|grep "ERROR"
if [ $? -eq 0] && [ !error ]; then 

After studying a bit about $? and shell scripting I know this wont work, what should I have to do to achieve this thing?

tripleee
  • 175,061
  • 34
  • 275
  • 318
Raaz444
  • 85
  • 1
  • 7
  • 1
    FYI, there are a lot of bad practices unrelated to your immediate question here. You'd be well-advised to read [Using Find](http://mywiki.wooledge.org/UsingFind) in full, and replace the `for t in $tests` hackery with a more robust practice. Running your code through http://shellcheck.net/ is also well-advised; likewise, reading through [BashPitfalls](http://mywiki.wooledge.org/BashPitfalls). – Charles Duffy Jan 30 '19 at 20:17

2 Answers2

2

The for loop for variable in one two three will run the body of the loop with $variable set to one on the first iteration, two on the next, and then to three. When the loop finishes, the variable retains the last value it had in the loop.

sed is a scripting language in its own right, though typically it is only used for trivial string substitutions. The sed script s/regex/replacement/g will replace every match on the regular expression with the static string replacement in the file(s) it processes. (In the absence of the /g flag, only the first occurrence on each input line will be replaced.)

Generally, scripts should examine the exit status from a command, not grep for human-readable strings. The make command already illustrates this, though it is not entirely idiomatic.

Incidentally, the shebang on the first line must start with exactly the two characters #!

Here is a refactoring with comments.

#!/usr/bin/env bash
#set -e

# Use lower case for your private variables
# BASEDIR was never used for anything
# Consistently use modern command substitution syntax
reporoot=$(git rev-parse --show-toplevel)

RED="$(tput setaf 1)"
GREEN="$(tput setaf 2)"
BLUE="$(tput setaf 4)"
NORM="$(tput sgr0)"

Test() {
    # Maybe don't pushd; see below
    # Quote the variable properly 
    pushd "$reporoot/rr/Fmda/body" > /dev/null
    # Refactor to avoid useless use of grep
    tests=$(find . -type f | sed -n "s/test\.xml//p")
    for t in $tests; do
      # Proper quoting; refactor to a single sed script
      # Fix sed syntax error (too many slashes)
      testname=$(echo "$t" | sed "s/\.\//;s%/test/%%")
      echo -e "${BLUE}EXECUTING $testname TEST${NORM}"
      pushd "$t" > /dev/null
      # Use "if" like it was meant to
      # Add grep condition
      if make test_svp && ! grep -q "ERROR" trace.log; then
        echo -e "${GREEN}SUCCESS IN $testname TEST MOVING ON!${NORM}"
      else
        echo "${RED}FAILURE IN $testname TEST${NORM}"
        exit 1
      fi
      popd >/dev/null
    done
    popd >/dev/null
}

Notice the addition of the grep after the make. The && says "and" and the ! inverts the exit code from grep (so this is true if grep fails to find any matches). The -q suppresses the output of matches by grep and causes it to stop searching as soon as it finds the first match.

More tangentially, I would regard pushd and popd as intended for interactive use. Shell scripts usually just cd in a subshell and then when the subshell finishes, you are back in the directory where you started. But here, a more fundamental refactoring might be called for.

For a somewhat more complex but hopefully robust refactoring, maybe do

Test() {
    find "$reporoot/rr/Fmda/body" -name 'test.xml' \
        \( -execdir sh -c '
            # XXX TODO: maybe replace with parameter substitutions
            testname=$(echo "$1" | sed "s%/test\.xml$%%;s/\.\//;s%/test/%%")
            echo -e "${BLUE}EXECUTING $testname TEST${NORM}"
            make test_svp &&
            ! grep -q "ERROR" trace.log &&
            echo -e "${GREEN}SUCCESS IN $testname TEST MOVING ON!${NORM}" ||
            { 
              echo "${RED}FAILURE IN $testname TEST${NORM}"
              exit 1; }' _ {} \; -o -quit \)
}

... though the -quit predicate is very much a GNU find extension (stol... flattered from here).

This has some assumptions about your test file names which might not be correct. Is test.xml the actual file name of the test, or just a suffix (or even somewhere in the middle)?

tripleee
  • 175,061
  • 34
  • 275
  • 318
  • If I had more time I would look into getting rid of the first `cd` entirely and refactoring the `find` to search for files with that name and loop over the results without a variable capture. Probably the remaining `sed` could be replaced with shell-native parameter substitutions. – tripleee Jan 30 '19 at 19:54
  • thanks @tripleee for the answer it worked, yes test.xml is not the complete name it is test.ecl.xml, now i found out that trace.log present at the same level as Makefile is not the right one it doesn't contain all the trace levels, there is an another file called test_trace.log which is one levels down from where makefile exists and also want to printout all the matching lines or maybe 10 of them. – Raaz444 Jan 31 '19 at 13:14
  • [test] >ls Makefile api unit trace.log otherfiles [test/unit/]> ls test_trace.log otherfiles now i want to use this test_trace.log instead of trace.log how can i find that file test_trace.log and use it to print the lines having ERROR i thought of using string=$(find . -type f | sed -n "s/test\.trace\.log//p") but didn't quite know how to use that in the if loop and used grep ERROR test_trace.log after the else echo statement it didn't print them out how to do this? – Raaz444 Jan 31 '19 at 13:14
  • Probably post a new question with more details if you can't figure it out. – tripleee Jan 31 '19 at 13:15
0
log=file.log
if grep ERRORS "$log" ; then echo BIG FAILURE ; fi

Should work with any shell.

(Thanks to tripleee useful comments)

tripleee
  • 175,061
  • 34
  • 275
  • 318
Alain Merigot
  • 10,667
  • 3
  • 18
  • 31
  • No no no. You want `if grep ERROR "$log"; then echo BIG FAILURE`. See https://stackoverflow.com/questions/36313216/why-is-testing-to-see-if-a-command-succeeded-or-not-an-antipattern – tripleee Jan 30 '19 at 18:44
  • grep returns 0 on lines found and 1 when no lines found, so the test must be reversed. And I do not use $? as in your very interesting link, I know it can lead to problems. I am totally aware that i may not be the best solution. But unless I am missing something, I do not see why it could lead to an incorrect behavior. – Alain Merigot Jan 30 '19 at 19:14
  • I agree that this is only vaguely related, but capturing a number to a variable so you can see if it's zero just to see if `grep` matched is exactly the same sort of pretzel logic. The *purpose* of `if` is to check if something succeeded. If the command you run as the argument to `if` succeeds (returns a result code equal to zero) then the `then` branch is taken. – tripleee Jan 30 '19 at 19:17
  • See also [useless use of `wc`](http://www.iki.fi/era/unix/award.html#wc) – tripleee Jan 30 '19 at 19:20
  • 1
    As a further refinement, `if grep -q ERROR "$log"` and notice also the [proper use of quoting.](/questions/10067266/when-to-wrap-quotes-around-a-shell-variable) – tripleee Jan 30 '19 at 19:21
  • You are definitely right and your links are very interesting. According to your second second links, backticks could lead to real problems if the number of errors is large. I should remove my answer, but it would suppress your useful comments... – Alain Merigot Jan 30 '19 at 19:30
  • Feel free to edit it into shape. This should probably be marked as a duplicate but I have not had the time to identify a suitable one. – tripleee Jan 30 '19 at 19:32