25

How do I send a job to a specific node in PBS/TORQUE? I think you must specify the node name after nodes.

#PBS -l nodes=abc

However, this doesn't seem to work and I'm not sure why. This question was asked here on PBS and specify nodes to use

Here is my sample code

#!/bin/bash
#PBS nodes=node9,ppn=1,
hostname
date 
echo "This is a script"
sleep 20    # run for a while so I can look at the details
date

Also, how do I check which node the job is running on? I saw somewhere that $PBS_NODEFILE shows the details, but it doesn't seem to work for me.

Community
  • 1
  • 1
Ashwin
  • 577
  • 3
  • 6
  • 15
  • Have you tried removing the `#!/bin/bash`. I am wondering if bash interprets `#PBS` as a comment... – iamauser Aug 23 '13 at 01:28
  • I didnt try that. so what do you think I should write in the first line? – Ashwin Aug 23 '13 at 01:45
  • Also is there a command to know on which node a job is running, I mean to be displayed by echo. I know from the terminal I can do that using qstat -n – Ashwin Aug 23 '13 at 01:47
  • `qstat -n1` gives you the information on which node your jobs are running. – Pixelchemist Aug 23 '13 at 17:22
  • 1
    @iamauser: Yes, bash interprets `#PBS` as a comment, but `qsub` recognizes it as a directive. That's exactly why that syntax was chosen. Removing the `#!/bin/bash` won't help. – Keith Thompson Aug 23 '13 at 19:05

2 Answers2

35

You can do it like this:

#PBS -l nodes=<node_name>

You can also specify the number of processors:

#PBS -l nodes=<node_name>:ppn=X

Or you can request additional nodes, specified or unspecified:

#PBS -l nodes=<node_name1>[:ppn=X][+<node_name2...]

That gives you multiple specific nodes.

#PBS -l nodes=<node_name>[:ppn=X][+Y[:ppn=Z]]

This requests the specific node with X execution slots from that node, plus an additional Y nodes with Z execution slots each.

Edit: To simply request a number of nodes and execution slots per node:

PBS -l nodes=X:ppn=Y

NOTE: this is all for TORQUE/Moab. It may or may not work for other PBS resource managers/schedulers.

Community
  • 1
  • 1
dbeer
  • 6,963
  • 3
  • 31
  • 47
  • 1
    This answer is not applicable to PBS Pro > v10 or v11. PBS Pro has changed the syntax for this type of selection. See Ch 5 of the User Guide for info. I'm only getting familiar with it myself so don't want to try to do it justice here. – dtlussier Nov 07 '14 at 15:41
  • 1
    True, this answer is meant specifically for TORQUE. – dbeer Nov 07 '14 at 18:07
  • How can I know what PBS resource manager my cluster has? (Without asking the admin, is there a command at the terminal that I can use?) – a06e Mar 26 '15 at 12:52
  • 1
    qstat --version might help. PBSPro versions are much higher than Torque versions. – dbeer Mar 26 '15 at 21:38
  • Do you know how to submit to a specific node in loadleveler? – Nike Dec 30 '18 at 18:32
4

The above answer doesn't work for PBS Pro. The following works for including a list of nodes (node1 and node2). 

#PBS -l select=1:host=node1+1:host=node2

For also including the number of processors,

#PBS -l select=1:ncpus=24:host=node1+1:ncpus=24:host=node2
rashid
  • 644
  • 7
  • 18
  • I am getting "Job violates queue and/or server resource limits" on PBS Pro 14.0.1. Any idea what this means? – soungalo Mar 08 '20 at 12:08
  • @soungalo It usually means the queue in which you are running the job doesn't allow you to submit jobs with the resources you have specified. You can get the various queues available and corresponding resource limits of the cluster you use by typing `qstat -Qf` in terminal. And you can specify the queue in which a job should run, by adding a line `#PBS -q queuename` in your jobscript. – rashid Apr 15 '20 at 04:19