Try catch in Nextflow processes

Question

How can I perform a try catch in nextflow?

I am currently writing a pipeline where it is possible that the bash command I am executing exits with an exitcode 1 under certain conditions. This brings my pipeline to a grinding halt. I would now like to use a try catch clause to define some alternative behavior in case this happens.

I have tried doing this in groovy fashion which does not seem to work:

process align_kallisto {

    publishDir "${params.outdir}/kallisto", mode: 'copy', saveAs:{ filename -> "${name}_abundance.tsv" }   

    input:
    tuple val(name), file(fastq) from fq_kallisto.dump(tag: 'kallisto fq')
    file(index) from kallisto_index.collect().dump(tag: 'kallisto index')

    output:
    file("output/abundance.tsv") into kallisto_quant

    // this can throw an exit 1 status
    try {
        """
        kallisto quant -i ${index} --bias --single --fr-stranded -o output --plaintext \
          --fragment-length ${params.frag_length} --sd ${params.frag_deviation} ${fastq}
        """
    } 
    // if this happens catch and do something else
    catch (Exception e) {
        println("Exception: ${e} for $name")
        """
        // execute some alternative command
        """
    }

}

Any advise?

I could tell nextflow to just ignore this error and still continue, but I would rather learn how to do a proper try catch.

Steve · Accepted Answer · 2021-02-11T23:29:49.177

AFAIK there's no way to handle errors in your process definition using a try/catch block. Rather than trying to catch all of the scenarios that result in an exit status 1, could you better define those conditions and handle them before trying to execute your process? For example, if an empty FASTQ file (or a FASTQ file with an insufficient number of reads as required by your process) was supplied as input and this resulted in an exit status 1, a pre-processing command that filtered out those files could be useful here.

But if it's not possible to better define the condition(s) that your command produces exit status 1 or any non-zero exit status, you can ignore them like you have suggested by appending errorStrategy 'ignore' to your process definition. Below is an example of how you could get the 'success' and 'failed' outputs, so they can be handled appropriately:

nextflow.enable.dsl=2

process test {

    errorStrategy 'ignore'

    input:
    tuple val(name), path(fastq)

    output:
    tuple val(name), path("output/abundance.tsv")

    """
    if [ "${fastq.baseName}" == "empty" ]; then
        exit 1
    fi
    mkdir output
    touch output/abundance.tsv
    """
}

workflow {

    fastqs = Channel.fromFilePairs( './data/*.fastq', size: 1 )

    test(fastqs) \
        .join(fastqs, remainder: true) \
        .branch { name, abundance, fastq_tuple ->
            failed: abundance == null
                return tuple( name, *fastq_tuple )
            succeeded: true
                return tuple( name, abundance )
        } \
        .set { results }

    results.failed.view { "failed: $it" }
    results.succeeded.view { "success: $it" }
}

Run with:

mkdir data
touch data/nonempty.fastq
touch data/empty.fastq

nextflow run -ansi-log false test.nf

Results:

N E X T F L O W  ~  version 20.10.0
Launching `test.nf` [suspicious_newton] - revision: b883179718
[08/60a99f] Submitted process > test (1)
[42/358d60] Submitted process > test (2)
[08/60a99f] NOTE: Process `test (1)` terminated with an error exit status (1) -- Error is ignored
success: [nonempty, /home/user/working/stackoverflow/66119818/work/42/358d60bd7ac2cd8ed4dd7aef665d62/output/abundance.tsv]
failed: [empty, /home/user/working/stackoverflow/66119818/data/empty.fastq]

Try catch in Nextflow processes

1 Answers1