2

I'm trying to use integer variables within a bash script for submitting to Sun Grid Engine (using qsub). Can someone clarify why this does not work?

NUMCORES=32

#$ -pe mpi $NUMCORES

(gives the error "Unable to read script file because of error: Numerical value invalid! The initial portion of string "$NUMCORES" contains no decimal number)

But this does:

#$ -pe mpi 32

I've looked at some solutions which involve awk or bc but can't seem to make them work. Obviously I am new to bash scripting - thanks for any help!

notadoctorthathelpspeople
  • 53
  • 8
  • Is it really a bash script? What is the first line? What do you mean by *"the initial portion of $NUMCORES contains no decimal number"*? Does the subsequent portion contain decimals? You show it as containing precisely `32` or are you not being wholly truthful? – Mark Setchell Jul 21 '16 at 21:21
  • 1
    Same script, or are you missing a `export NUMCORES=32`? – Diego Torres Milano Jul 22 '16 at 03:11

1 Answers1

1

The character # in a bash script signals comments. Hence, in the line #$ -pe mpi $NUMCORES, the value of the environement variable NUMCORES will not be used, since it is a comment.

Yet, these comments are useful to pass parameters to the qsub command. See the man page and these examples for instance. Notice that these parameters can be passed on the command line:

qsub -pe mpi 32 myjob.sh

The alternative is to use a command before qsub to generate the job. Let's imagine that the job script jobgeneric.sh is:

#!/bin/bash
#$ -N useless
#$ -cwd
#$ -pe mpi $NUMCORES
#$ -l h_vmem=6G

echo 42

Let's use the sed command to substitute the environment variable and create a new file job.sh:

export NUMCORES=32
sed 's/$NUMCORES/'$NUMCORES'/g' jobgeneric.sh > job.sh
cat job.sh

See also Environment variable substitution in sed . The result is:

#!/bin/bash
#$ -N useless
#$ -cwd
#$ -pe mpi 32
#$ -l h_vmem=6G

echo 42

Now the job can be submitted by using qsub job.sh!

Community
  • 1
  • 1
francis
  • 9,525
  • 2
  • 25
  • 41
  • Thanks francis. I ended up taking a similar approach using python to handle specifying cores and other parameters prior to generating the script. Then the script gets automatically submitted to the queue. – notadoctorthathelpspeople Jul 26 '16 at 15:29