3

This is an extension of a previous question I asked:

Using name of BASH script as input argument

My goal is to write a BASH script which takes the arguments from the file's name and uses them as inputs for a Julia code I'm writing, and then submit the BASH script to a remote cluster. Using @AndriyMakukha's solution, I was able to write the following script through Torque:

#!/bin/bash
 
#PBS -l mem=10gb,nodes=1:ppn=2,walltime=1:00:00
#PBS -N ES_100_20_100
#PBS -j oe
#PBS -o ./log/julia.${PBS_JOBID}.out

module load julia/1.5.1 python/3.8.1
 
PARAM1=$( echo ${0%%.pbs} | awk -F "_" '{print $2}' )  
PARAM2=$( echo ${0%%.pbs} | awk -F "_" '{print $3}' )  
PARAM3=$( echo ${0%%.pbs} | awk -F "_" '{print $4}' )  
PARAM4=$( echo ${0%%.pbs} | awk -F "_" '{print $5}' ) 
PARAM5=$( echo ${0%%.pbs} | awk -F "_" '{print $6}' )  
 
echo "Param 1: $PARAM1"
echo "Param 2: $PARAM2"
echo "Param 3: $PARAM3"
echo "Param 4: $PARAM4"
echo "Param 5: $PARAM5"

cd path/to/code 
julia Example.jl $PARAM1 $PARAM1 $PARAM2 $PARAM3 $PARAM4 $PARAM5

This script (called "Example_1_2_3_4_5.pbs") prints the different parameters from the filename to the output file, and then runs the Julia code with said parameters as the ARGS. When I run this on the local machine, it works great. When I submit the code to the cluster via qsub, I get the following error in the output file:

Param 1: priv/jobs/3574314-1.orion.cm.cluster.SC
Param 2:
Param 3:
Param 4:
Param 5:
ERROR: LoadError: ArgumentError: invalid base 10 digit 'p' in "priv/jobs/3574314-1.cluster_name.cm.cluster.SC"

Obviously, the code isn't reading the parameters correctly; i.e., it returns the name of the job, not the name of the BASH file itself. This is obvious because

echo ${0%%.pbs}

returns

/cm/local/apps/torque/var/spool/mom_priv/jobs/3574314-1.orion.cm.cluster.SC

How can I get the name of the pbs file itself if I submit to cluster, seeing as ${0%%.pbs} doesn't work?

Joshuah Heath
  • 663
  • 1
  • 5
  • 20
  • What do you see if you add `echo "$*"` at the beginning of your script? Can you elaborate on what you mean by "submit the code to the cluster via qsub"? – Shane Bishop Jan 16 '21 at 23:29
  • @ShaneBishop It doesn't print anything when I try that. I mean when I just run "qsub script_name.pbs", I get the error described above. When I just run "bash script_name.pbs", it runs fine. – Joshuah Heath Jan 16 '21 at 23:36
  • 1
    What about using `-v variable_list` option of qsub and adding those variables yourself when submitting the job? – Przemyslaw Szufel Jan 16 '21 at 23:41
  • 1
    Have you looked at https://stackoverflow.com/questions/26487658/does-qsub-pass-command-line-arguments-to-my-script/26513592? You can use `-F` or, if `-F` is not available, it might be possible with `-v`. – Shane Bishop Jan 16 '21 at 23:44
  • @ShaneBishop There are no command line arguments. The script interprets a part of its filename as "arguments". – Socowi Jan 16 '21 at 23:51
  • 1
    @Socowi yes but perhaps you could have a "proxy" bash file that would check the filename and than build a `qsub` command with the `-v` parameter? – Przemyslaw Szufel Jan 17 '21 at 01:45
  • @PrzemyslawSzufel Perhaps I can restructure the above .pbs file so that the final line is "julia Example.jl x y z t r", and then submit via "qsub -F "x=1 y=2 z=3 t=4 r=5" test.pbs"? Would this be the correct syntax, or would I have to use -v? Then, as you said, I could make a "proxy" file that just runs qsub 50+ times with the different input parameters I need. – Joshuah Heath Jan 17 '21 at 02:02
  • again I do not have pbs installation to test but I first would try `-v` and read the params in julia from `ENV` variable – Przemyslaw Szufel Jan 17 '21 at 10:27

0 Answers0