Override variables in AWK scripts in command line/mother scripts

Question

If we have an AWK script (average.sh) like this and would like to use it to process a lot of inputfiles:

awk -F\" 'BEGIN{print}
  last != $4""$8 && last{
      print line,exp(C/D)
      C=D=0}
  { # This block process each line of infile
   C += log($(NF-1)+0)
   D++
   $(NF-1)=""
   line=$0
   last=$4""$8}
  END{ # This block triggers after the complete file read
       # to print the last average that cannot be trigger during
       # the previous block
      print line,exp(C/D)}' ${var2:=infile}

Now if we do

export var2="infile4" | sh average.sh

The "average.sh" still process "infile" instead of "infile4".

Following the best answers in Override a variable in a Bash script from the command line , we try

var2=infile4 ./geometric_average_real

This give an error of "var2=infile: Command not found."

Our final goal is to write loops

for (X=1; X<=3; X++)

do

sh average.sh infile${X}

done

so this loop shall process

sh average.sh infile1
sh average.sh infile2
sh average.sh infile3

`export var2="infile4" | sh average.sh` should not use a pipe. That should be a semicolon separating the commands. The pipe is causing you problems because you are spawning sub-shells. Try without that. — Etan Reisner, Jun 27 '14 at 19:26
`var2=infile4 ./geometric_average_real` should work fine in `bash` but might not work in other shells (almost certainly not in `sh`). Make sure you are trying with `bash` and not `sh` if you want `bash` semantics. — Etan Reisner, Jun 27 '14 at 19:27
Using an argument like your goal should work fine if you use the positional parameter arguments in your shell script instead of named variables (like `var2`). Did you try that and have it not work somehow? — Etan Reisner, Jun 27 '14 at 19:27
By the way, you don't need to write `$4""$8`. `$4 $8` will work just fine, as will `$4$8`. If `a` and `b` are two variables, `a b` is their string concatenation. Just watch out for awk's odd expression precedence for the invisible concatenation operator. — rici, Jun 27 '14 at 23:04

score 1 · Accepted Answer · answered Jun 27 '14 at 19:51

1

If your average.sh script is working for a single infile, then just call it from another script with a list of files you would like to process:

#!/bin/bash

test -n "$1" || { echo "error, insufficient input"; exit 1; }

avgscript="/path/to/average.sh"

test -x "$avgscript" || { echo "error: required script `$avgscript` not found or not executable"; exit 1; }

for i in "$@"; do

    avgscript "$i"

done

Save that file as say runavg.sh and then simply call runavg.sh file1 file2 file2

answered Jun 27 '14 at 19:51

David C. Rankin

81,885
6
58
85

Hi David, thanks for the comment! However, if we don't put a file name right after .....print line,exp(C/D)}' ,there will be an error message "awk: cmd. line 14 ^ unexpected newline or end of string". I use "sh average.sh input.csv" in bash – Chubaka Jun 27 '14 at 21:17
Well, darn. Can you put you entire awk script inside the above script where it says `avgscript` in the `for` loop in `$( your awk script "$i" )` with I in double-quotes and make the substitution work? – David C. Rankin Jun 27 '14 at 21:58
`for i; do` will also loop over all the positional arguments. – Etan Reisner Jun 27 '14 at 23:06

Override variables in AWK scripts in command line/mother scripts

1 Answers1