Stack Overflow
Stack Overflow

Questions tagged [lsf]

LSF, aka Load Sharing Facility is software that executes batch jobs on networked Unix and Windows systems on many different architectures. It is commonly used in HPC Clusters in many universities and research centers around the world.

The Load Sharing Facility or LSF is a set of tools for distributing jobs across a set of networked systems. If was developed initially by the Platform Computing company, which was acquired by IBM in 2012. It is now called IBM Spectrum LSF

Resources:

229 questions
16
votes
2 answers

How can I get detailed job run info from SLURM (e.g. like that produced for "standard output" by LSF)?

When using bsub with LSF, the -o option gave a lot of details such as when the job started and ended and how much memory and CPU time the job took. With SLURM, all I get is the same standard output that I'd get from running a script without LSF. For…
Christopher Bottoms
  • 11,218
  • 8
  • 50
  • 99
14
votes
3 answers

SLURM display the stdout and stderr of an unfinished job

I used to use a server with LSF but now I just transitioned to one with SLURM. What is the equivalent command of bpeek (for LSF) in SLURM? bpeek bpeek Displays the stdout and stderr output of an unfinished job I couldn't find the documentation…
Dnaiel
  • 7,622
  • 23
  • 67
  • 126
12
votes
5 answers

LSF - Get ID of submitted job

Say I submit a job using something like bsub pwd. Now I would like to get the job ID of that job in order to build a dependency for the next job. Is there some way I can get bsub to return the job ID?
Nils
  • 1,936
  • 3
  • 27
  • 42
10
votes
5 answers

Can I suppress an LSF job report without sending mail?

I would like to submit a job with Platform LSF and have the output placed in a file (bsub -o), without a job report at the end of it. Using bsub -N removes the job report from the file, but instead sends the report via e-mail. Is there a way to…
Michael Hoffman
  • 32,526
  • 7
  • 64
  • 86
9
votes
3 answers

Is there a "one-liner" for submitting many jobs to SLURM (similar to LSF)?

Can I submit "one-liners" to SLURM? Using bsub from LSF and the standard Linux utility xargs, I can easily submit a separate job for uncompressing all of the files in a directory: ls *.gz | sed 's/.gz$//g' | xargs -I {} bsub 'gunzip -c {}.gz >…
Christopher Bottoms
  • 11,218
  • 8
  • 50
  • 99
8
votes
1 answer

LSF (bsub): how to specify a single "wrap-up" job to be run after all others finish?

BASIC PROBLEM: I want to submit N + 1 jobs to an LSF-managed Linux cluster in such a way that the (N + 1)-st "wrap-up" job is not run until all the preceding N jobs have finished. EXTRA: If possible, it would be ideal if I could arrange matters so…
kjo
  • 33,683
  • 52
  • 148
  • 265
7
votes
1 answer

How can I run multiple jobs from a large queue at the same time in Perl?

I would like to program job limits for the LSF command bsub into my Perl script which launches LSF jobs under the hood. If I have something like 2000 jobs, I would like to run at most 20 jobs at any given time. I have seen scripts that launch 20…
Gordon
  • 1,633
  • 3
  • 26
  • 45
6
votes
2 answers

Why are repetitive calls to squeue in Slurm frown upon?

Why is it not recommended to run squeue in a loop to avoid overloading Slurm, but no such limitations are mentioned for the bjobs tool from LSF or qstat from SGE ? The man page for squeue states: PERFORMANCE Executing squeue sends a remote…
E. Morice
  • 63
  • 3
6
votes
2 answers

Error with LSF Platform: lsb_init: Failed in an LSF library call: Unable to open file lsf.conf

I have an issue with LSF Platform I cannot wrap my head around. For scripting reason, I need to check the running/pending jobs with 'bjobs' (and other b***) with a perl script. For some reason it did not work, and I was able to view the following…
user1921895
  • 65
  • 1
  • 4
5
votes
2 answers

automatically retrieve results of bsub

I am looking for some general advice rather than a coding solution. Basically when submitting a job via bsub I can retrieve a log of the Stdin/Stdout by specifying any of the following: bsub -o log.txt % sends StdOut to log.txt bsub -u me@email…
brucezepplin
  • 9,202
  • 26
  • 76
  • 129
5
votes
1 answer

how to handle quotes and semicolon in bsub command

I'm trying to submit commands to the LSF scheduler with bsub but this command includes a parameter value that must be quoted and contains a semicolon. Here is a simple command to illustrate my problem bsub -o t.o -e t.e echo "foo;bar" it fails…
tospo
  • 646
  • 9
  • 17
4
votes
1 answer

Hints for custom bash completion

I am developing a custom bash completion command to capture job IDs from a scheduling system (LSF, PBS, SLURM). I've got the basic functionality, but I would like to extend it now with "hints" that I've seen when running zsh. For instance when I…
Robert Manson-Sawko
  • 364
  • 4
  • 18
4
votes
0 answers

how to realize an automatic bpeek?

I am on a cluster equiped with IBM LSF job system. I want to realize a feature, so that I can peek current output of every job I am running So I write for((i=2;i<=$(bjobs|grep RUN|wc -l)+1;i+=1)); do bpeek -f $(bjobs | awk '{print $1}'| head -n…
user15964
  • 2,507
  • 2
  • 31
  • 57
4
votes
1 answer

Get CWD of bjobs in LSF environment

bjobs -l gives the long description of the job where CWD is split across say 3 lines. I want a command that can reliable fetch me CWD.
user5005786
  • 65
  • 2
3
votes
3 answers

Checking the position of my jobs within a queue in LSF

Is there a way to check the position of my jobs within an LSF queue? If I run: bjobs -u all -q my_queue I get a list of jobs from all users within my_queue, but is this list sorted by the position of my jobs within the queue?
Amelio Vazquez-Reina
  • 91,494
  • 132
  • 359
  • 564
1
2 3
…
15 16