Stack Overflow
Stack Overflow

Questions tagged [qsub]

Qsub is a job submission command for high performance computing jobs.

Qsub is a job submission command for high performance computing jobs. It is used by some resource managers and some schedulers, a few of which are TORQUE, PBSPro, OpenPBS, and SGE. It has many different options and is the way that a job (work request) gets queued for consideration in the cluster.

Questions with the "qsub" tag should clearly indicate which resource manager or scheduler is being used.

385 questions
49
votes
4 answers

parameter for shell scripts that is started with qsub

how can I parametrize a shell script that is executed on a grid (started with qsub) ? I have a shell script, where I use getopts to read the parameters. When I start (qsub script.sh -r firstparam -s secondparam ..) this working script with qsub I…
Martin
  • 751
  • 1
  • 5
  • 8
42
votes
9 answers

Wait for set of qsub jobs to complete

I have a batch script which starts off a couple of qsub jobs, and I want to trap when they are all completed. I don't want to use the -sync option, because I want them to be running simultaneously. Each job has a different set of command line…
Nupur
  • 1,051
  • 6
  • 15
  • 20
33
votes
3 answers

how to specify error log file and output file in qsub

I have a qsub script as #####----submit_job.sh---##### #!/bin/sh #$ -N job1 #$ -t 1-100 #$ -cwd SEEDFILE=/home/user1/data1 SEED=$(sed -n -e "$SGE_TASK_ID p" $SEEDFILE) /home/user1/run.sh $SEED The problem is-- it puts…
d.putto
  • 7,185
  • 11
  • 39
  • 45
21
votes
3 answers

excluding nodes from qsub command under sge

I have more than 200 jobs I need to submit to and sge cluster. I'll be submitting them into two ques. One of the ques have a machine that I don't want to submit jobs to. How can I exclude that machine? The only thing I found that might be helpful is…
Yotam
  • 10,295
  • 30
  • 88
  • 128
19
votes
2 answers

How to Pass Parameters from QSub to Bash Script?

I'm having an issue passing variables to a Bash script using QSub. Assume I have a Bash script named example. The format of example is the following: #!/bin/bash # (assume other variables have been set) echo $1 $2 $3 $4 So, executing "bash…
user2800589
  • 191
  • 1
  • 1
  • 3
18
votes
3 answers

Does qsub pass command line arguments to my script?

When I submit a job using qsub script.sh is $@ setted to some value inside script.sh? That is, are there any command line arguments passed to script.sh?
a06e
  • 18,594
  • 33
  • 93
  • 169
14
votes
4 answers

Submit jobs to a slave node from within an R script?

To get myscript.R to run on a cluster slave node using a job scheduler (specifically, PBS) Currently, I submit an R script to a slave node using the following command qsub -S /bin/bash -p -1 -cwd -pe mpich 1 -j y -o output.log ./myscript.R Are…
David LeBauer
  • 31,011
  • 31
  • 115
  • 189
13
votes
3 answers

Directly pass parameters to pbs script

Is there a way to directly pass parameters to a .pbs script before submitting a job? I need to loop over a list of files indicated by different numbers and apply a script to analyze each file. The best I've been able to come up with is the…
Sevenless
  • 2,805
  • 4
  • 28
  • 47
11
votes
1 answer

QSUB: Specify output and error files for each task in Job Array

Hopefully this is not a dublicate and also not just a problem of our cluster's configuration... I am submitting a job array to a cluster using qsub with the following command: qsub -q QUEUE -N JOBNAME -t 1:10 -e ${ERRFILE}_$SGE_TASK_ID…
niak
  • 340
  • 3
  • 11
11
votes
1 answer

How to use the qsub -v command in PBS torque?

I would like to pass variables to a csh script by using "qsub -v" command. I understand we can list the parameters-value pairs as below, qsub -v par1=value1 par2=value2 myScript.csh Does anyone know if the values of these parameters can be a…
Cassie
  • 1,179
  • 6
  • 18
  • 30
11
votes
6 answers

'Unexpected end of file' and 'error importing function definition' error running shellscript using qsub

I have the following shellscript: #!/bin/sh cd /sw/local/bin/ export LD_LIBRARY_PATH=/sw/local/lib:/usr/local/Trolltech/Qt-4.7.2/lib:$LD_LIBRARY_PATH ./FeatureFinderRaw -in /homes/JG-C1-18.mzML -out /homes/test_remove_after_use.featureXML -threads…
Niek de Klein
  • 8,524
  • 20
  • 72
  • 143
10
votes
1 answer

Python: PBS submission, what happens if I change script?

I have a script job.bin for executing a code in python (pythoncode.py) on a cluster, submitting jobs via qsub. In job.bin: cd /path/to/my/python/code python pythoncode.py What happens if: a) while pythoncode.py is running, I change its content b)…
johnhenry
  • 1,293
  • 5
  • 21
  • 43
10
votes
4 answers

"embarrassingly parallel" programming using python and PBS on a cluster

I have a function (neural network model) which produces figures. I wish to test several parameters, methods and different inputs (meaning hundreds of runs of the function) from python using PBS on a standard cluster with Torque. Note: I tried…
meduz
  • 3,903
  • 1
  • 28
  • 40
10
votes
2 answers

how to limit number of concurrently running PBS jobs

I have a 64-node cluster, running PBS Pro. If I submit many hundreds of jobs, I can get 64 running at once. This is great, except when all 64 jobs happen to be nearly I/O bound, and are reading/writing to the same disk. In such cases, I'd like to…
Aaron
  • 121
  • 1
  • 6
10
votes
2 answers

what is 'Gbytes seconds'?

From the qstat (Sun Grid Engine) manpage: mem: The current accumulated memory usage of the job in Gbytes seconds. What does that mean?
Reactormonk
  • 21,472
  • 14
  • 74
  • 123
1
2 3
…
25 26