Use of a HEREDOC with SLURM sbatch --wrap

Question

I am encountering difficulties using a (Bash) HEREDOC with a SLURM sbatch submission, via --wrap.

I would like the following to work:

SBATCH_PARAMS=('--nodes=1' '--time=24:00:00' '--mem=64000' '--mail-type=ALL')

sbatch ${SBATCH_PARAMS[@]} --job-name="MWE" -o "MWE.log" --wrap <<EOF
        SLURM_CPUS_ON_NODE=\${SLURM_CPUS_ON_NODE:-8}
        SLURM_CPUS_PER_TASK=\${SLURM_CPUS_PER_TASK:-\$SLURM_CPUS_ON_NODE}
        export OMP_NUM_THREADS=\$SLURM_CPUS_PER_TASK

        parallel --joblog "MWE-jobs.log" --resume --resume-failed -k --linebuffer -j \$((\$OMP_NUM_THREADS/4)) --link "MWE.sh {1} {2}" ::: "./"*R1*.fastq.gz ::: "./"*R2*.fastq.gz
EOF

On my current cluster, sbatch returns the below error, refusing to submit this job:

ERROR:   option --wrap requires argument

Might anyone know how I can get this to work?

score 1 · Accepted Answer · answered Oct 30 '18 at 21:09

Since wrap expects a string argument, you can't use a heredoc directly. Heredocs are used when a filename is expected where it's undesirable to make one.

Use a heredoc for cat, where it does expect a filename, and use its output as the string for which --wrap expects:

SBATCH_PARAMS=('--nodes=1' '--time=24:00:00' '--mem=64000' '--mail-type=ALL')

sbatch ${SBATCH_PARAMS[@]} --job-name="MWE" -o "MWE.log" --wrap $(cat << EOF
        SLURM_CPUS_ON_NODE=\${SLURM_CPUS_ON_NODE:-8}
        SLURM_CPUS_PER_TASK=\${SLURM_CPUS_PER_TASK:-\$SLURM_CPUS_ON_NODE}
        export OMP_NUM_THREADS=\$SLURM_CPUS_PER_TASK

        parallel --joblog "MWE-jobs.log" --resume --resume-failed -k --linebuffer -j \$((\$OMP_NUM_THREADS/4)) --link "MWE.sh {1} {2}" ::: "./"*R1*.fastq.gz ::: "./"*R2*.fastq.gz
EOF)

I wish I could. I've never found it, if you do, please let me know. See [here](https://www.tldp.org/LDP/abs/html/here-docs.html) and [here](https://stackoverflow.com/questions/2953081/how-can-i-write-a-heredoc-to-a-file-in-bash-script). `<<<` are here strings which have a similar functionality without a delimiter. — Rafael, Oct 30 '18 at 22:44

score 1 · Answer 2 · answered Oct 30 '18 at 22:47

1

You can just use the heredoc without the wrap provided you add the #!/bin/bash at the top of it.

answered Oct 30 '18 at 22:47

Carles Fenoy

4,740
1
26
27

score 0 · Answer 3 · answered Oct 30 '18 at 19:55

Adapting a related post on assigning a HEREDOC to a variable, but instead using cat (since I use errexit and want to avoid working-around the non-zero exit value of the read), I was able to submit my job as follows:

CMD_FOR_SUB=$(cat <<EOF
    SLURM_CPUS_ON_NODE=\${SLURM_CPUS_ON_NODE:-8}
    SLURM_CPUS_PER_TASK=\${SLURM_CPUS_PER_TASK:-\$SLURM_CPUS_ON_NODE}
    export OMP_NUM_THREADS=\$SLURM_CPUS_PER_TASK

    parallel --joblog "MWE-jobs.log" --resume --resume-failed -k --linebuffer -j \$((\$OMP_NUM_THREADS/4)) --link "MWE.sh {1} {2}" ::: "./"*R1*.fastq.gz ::: "./"*R2*.fastq.gz
EOF
)

sbatch ${SBATCH_PARAMS[@]} --job-name="MWE" -o "MWE.log" --wrap "$CMD_FOR_SUB"

While this does appear to work, I would still prefer a solution that allows sbatch to directly accept the HEREDOC.

Use of a HEREDOC with SLURM sbatch --wrap

3 Answers3