Bash variable caused syntax error inside awk

Question

I got the following error

awk: cmd. line:1: {if(NR%4==2) print length(.)}
awk: cmd. line:1:                           ^ syntax error

While running sh s.sh .

#!/bin/bash
#usage: sh afterqc_pbs.sh /work/waterhouse_team/All_RawData/Banana_Each_Cell_Raw/Banana_Illumina/QUT_WT_GN366_4 

for r1 in $(find $1 -name "*R1*.gz");
do
  base=${r1%_R1*}
  echo $base
  output=$(basename $(echo $r1 | sed 's/R1//g'))
  r2=$(echo $r1 | sed 's/R1/R2/g')
  echo $r1
  echo $r2 


  #cat <<EOF
  qsub <<EOF
#!/bin/bash -l

#PBS -N $output
#PBS -l walltime=48:00:00
#PBS -j oe
#PBS -l mem=50G
#PBS -l ncpus=1
#PBS -M m.lorenc@qut.edu.au
##PBS -m bea

cd \$PBS_O_WORKDIR

echo "Average\n"
zcat $r1 | awk '{if(NR%4==2) {count++; bases += length} } END{print bases/count}'

echo "Distribution\n"
zcat $r1 | awk '{if(NR%4==2) print length($1)}' | sort -n | uniq -c


EOF

done

How is it possible that awk does not picks up . rather than the length?

Thank you in advance.

As a start, try writing your heredoc output to a file instead of immediately submitting it to `qsub`. Just reviewing the generated source may help. Likely issue is quoting after `$1` is substituted. — Will Bickford, Sep 06 '18 at 01:53
You might want to paste your script at http://shellcheck.net/ and fix the errors it reports. A complete refactoring would be an improvement, but that's a start. — tripleee, Sep 06 '18 at 04:15

Joseph Sible-Reinstate Monica · Answer 1 · 2018-09-06T02:13:11.880

2

The issue is your use of qsub <<EOF. When you use << (called a "here document" or "heredoc"), any variables inside get expanded. The single quotes around them don't count, as the heredoc takes priority. If you want to change this, change <<EOF to <<"EOF". https://www.gnu.org/software/bash/manual/bashref.html#Here-Documents has more details.

edited Sep 06 '18 at 02:13

answered Sep 06 '18 at 01:54

Joseph Sible-Reinstate Monica

45,431
5
48
98

Could you please provide where the quotes in EOF have to be set? – user977828 Sep 06 '18 at 02:07
Around the string EOF itself. I edited my answer to clarify this. – Joseph Sible-Reinstate Monica Sep 06 '18 at 02:09
however, you don't need/want dbl-quotes on your closing `EOF`. Good luck to all. – shellter Sep 06 '18 at 02:12
I changed it but now I got `qsub: illegal -N value`. What did I miss? – user977828 Sep 06 '18 at 03:38
-N lets you specify a name to qsub. You're not passing one, so it's trying to make one up, but the one it's making up is invalid. Try passing it a simple one you know is valid. – Joseph Sible-Reinstate Monica Sep 06 '18 at 03:50
In the 5th line `output=$(basename $(echo $r1 | sed 's/R1//g'))` – user977828 Sep 06 '18 at 04:01
What about that line? – Joseph Sible-Reinstate Monica Sep 06 '18 at 04:30
the 5th line contains the input for `#PBS -N $output`. – user977828 Sep 06 '18 at 04:40
Okay, I see the issue there. Now that you're not substituting variables, that's the literal string $output, and not the value of the $output variable. – Joseph Sible-Reinstate Monica Sep 06 '18 at 04:50
I do not understand. How is it possible to fix it? – user977828 Sep 06 '18 at 05:09
The solution was `\$1` without `"EOF"`. – user977828 Sep 07 '18 at 03:48

Bash variable caused syntax error inside awk

1 Answers1