1

I am trying to execute a series of job in different directories. I want to pass the directory as an input argument to the job. So far I understood that I can use environmental variable as a way to send argument to the jobs. But the problem is since jobs run in parallel fashion, the last value of this variable will be used for all jobs. let look at my code :

for i in "${arr[@]}"
  do
  export dir=$i 
  qsub myBashFile.sh
done

and in my job I used the variable dir to do some operation. I want each job execute with its own input parameter. Edit: this is my job

#!/bin/sh
#
#
#PBS -N Brownie
#PBS -o test.output.txt
#PBS -e test.error.txt
#PBS -l walltime=2:00:00
#PBS -m n
#PBS -V dir
cd $dir
./run_mycode.sh

I know this is not correct, but i am looking for an alternative way to keep the value of dir unchanged and unique for all jobs independently.

I also tried to modify a variable in job file with sed command like below:

 sed "s/dir/"'$i'"/g" my_job.sh > alljobs/my_jobNew.sh

but instead of putting the actual value of $i, dir changes exactly to $i which is meaningless in my_job.sh.

user2373198
  • 147
  • 10

2 Answers2

0

Have you tried passing the directory as command_args as explained in the manpage qsub(1)? That would be:

for i in "${arr[@]}"
  do
  qsub myBashFile.sh -- "$i"
done

You should be able to access it as $1 inside myBashFile.sh.

Michael Jaros
  • 4,586
  • 1
  • 22
  • 39
  • If above does not work for you, maybe this [related question on StackOverflow](http://stackoverflow.com/questions/18925068/how-to-pass-parameters-from-qsub-to-bash-script) can help – Michael Jaros Mar 06 '15 at 09:23
  • I wrote a simple job to test if it works or not, but it didn't work . here is my simple job : #!/bin/sh #PBS -N test #PBS -o test.output.txt #PBS -e test.error.txt #PBS -l walltime=00:01:00 #PBS -m n echo $1 I call it like this: qsub job_test.sh "hello" and also this as you said: qsub job_test.sh .. "hello" – user2373198 Mar 06 '15 at 10:58
  • usage: qsub [-a date_time] [-A account_string] [-b secs] [-c [ none | { enabled | periodic | shutdown | depth= | dir= | interval=}... ] [-C directive_prefix] [-d path] [-D path] [-e path] [-h] [-I] [-j oe|eo|n] [-k {oe}] [-l resource_list] [-m n|{abe}] [-M user_list] [-N jobname] [-o path] [-p priority] [-P proxy_user [-J – user2373198 Mar 06 '15 at 11:03
  • You wrote `..`, but it should be `--`. – Michael Jaros Mar 06 '15 at 14:02
  • Also, have you checked out the link in my comment? There is another solution on that page. – Michael Jaros Mar 06 '15 at 14:03
  • yes I checked it , but as I mentioned in my question enviromental variable has a problem that in parallel fashion only the last value will be used – user2373198 Mar 06 '15 at 15:48
0

I would use $PBS_O_WORKDIR for this. Change your submission script to this:

for i in "${arr[@]}"
  do
  cd /path/to/$i 
  qsub /path/to/myBashFile.sh
done

In your job you would then change 'cd $dir' to 'cd $PBS_O_WORKDIR'.

chuck
  • 735
  • 3
  • 4
  • thanks for you replay.But I want to find a way to pass safely argument to job, it was a simple sample to define the problem in the question but i can not navigate to the destination folder, because I want to keep separate my binary files from my data sets, I mean I don't want to copy any exe file in my data files. – user2373198 Mar 06 '15 at 13:28