33

I have a qsub script as 

#####----submit_job.sh---#####
    #!/bin/sh
    #$ -N job1
    #$ -t 1-100
    #$ -cwd
    SEEDFILE=/home/user1/data1
    SEED=$(sed -n -e "$SGE_TASK_ID p" $SEEDFILE)
    /home/user1/run.sh $SEED

The problem is-- it puts all error and output files (job1.eJOBID & job1.oJOBID) to the same directory from where I am running qsub submit_job.sh while I want to save these file (output and error log file in same different place (specified as $SEED_output). I tried to change the line as

/home/user1/run.sh $SEED -o $SEED_output

But it didn't work. Any suggestion ?? How can I specify path and name of default output and error log file ??

d.putto
  • 7,185
  • 11
  • 39
  • 45
  • I had in slurm `#SBATCH --output="demo.%j.%N.out" #SBATCH --error="demo.%j.%N.err"`, how do I adapt it to qsub? – Charlie Parker Oct 20 '20 at 20:08
  • related: https://community.openpbs.org/t/how-to-include-the-job-id-and-other-info-in-the-output-file-for-qsub/2369 `How to include the job id and other info in the output file for qsub?` – Charlie Parker Nov 17 '20 at 21:44

3 Answers3

43

Typically error and output files are given as pbs directives in the qsub script or as command line options to the qsub script like so:

#! /bin/bash
#PBS -q queue_name
#PBS -A account_name
#PBS -l nodes=12:ppn=12
#PBS -l walltime=18:00:00
#PBS -e /mypath/error.txt
#PBS -o /mypath/output.txt

or as a command line option to qsub like so:

qsub -o /mypath/output.txt -e /mypath/error.txt submit_job.sh

With the first option I don't think you can use a variable as the shell won't look at lines that are commented. Plus I think PBS deals with the commented lines before the shell would. If you know the path when you're invoking qsub, you could try the second option. Alternatively you could try simply redirecting the output and error in the script itself:

/home/user1/run.sh $SEED > ${SEED}/output.txt 2> ${SEED}/error.txt

The third option is probably the easiest. Output and error files might still be created in the run directory, though they'd likely be empty.

John Dood
  • 113
  • 8
Andy
  • 644
  • 5
  • 9
2

At first glance, you need brackets around your variable in the -o declaration.

/home/user1/run.sh $SEED -o ${SEED}_output

Otherwise bash is looking for a variable called ${SEED_output} which doesn't exist.

Andy
  • 644
  • 5
  • 9
  • Thanks for mentioning that, but I still have no luck :( I tried /home/user1/run.sh $SEED -o ${SEED}_output and nothing changed. Just to make things clear $SEED (or ${SEED}) is path to a directory. So i think output should save in file _output in the directory ${SEED} !!! – d.putto Feb 02 '12 at 11:27
  • -o is passed to run.sh, so that won't work unless run.sh handles -o as a command line argument. – Andy Feb 02 '12 at 19:05
  • I had in slurm `#SBATCH --output="demo.%j.%N.out" #SBATCH --error="demo.%j.%N.err"`, how do I adapt it to qsub? – Charlie Parker Oct 20 '20 at 20:08
  • what if you want to include the jobid number and other info like that? – Charlie Parker Nov 17 '20 at 21:40
0

I ran into the same problem. And my workaround was to have:

#$ -N script-name
#$ -wd <path-to-project>/log 
cd ..

in the qsub script. Now the error and output file are saved in <path-to-project>/log and the script works in <path-to-project> At least for me that worked

L3000
  • 1