Is it possible to add a conditional statement in snakemake's rule all?

Question

I want to run multiple snakefiles called qc.smk , dada2.smk, picrust2.smkusing singularity. Then there is one snakefile called longitudinal.smk I would like to run conditionally. For example, if longitudinal data is being used.

# set vars

LONGITUDINAL = config['perform_longitudinal']

rule all:
  input:
    # fastqc output before trimming
    raw_html = expand("{scratch}/fastqc/{sample}_{num}_fastqc.html", scratch = SCRATCH, sample=SAMPLE_SET, num=SET_NUMS),
    raw_zip = expand("{scratch}/fastqc/{sample}_{num}_fastqc.zip", scratch = SCRATCH, sample=SAMPLE_SET, num=SET_NUMS),
    raw_multi_html = SCRATCH + "/fastqc/raw_multiqc.html",
    raw_multi_stats = SCRATCH + "/fastqc/raw_multiqc_general_stats.txt"

# there are many more files in rule all

##### setup singularity #####

singularity: "docker://continuumio/miniconda3"

##### load rules #####

include: "rules/qc.smk"
include: "rules/dada2.smk"
include: "rules/phylogeny.smk"
include: "rules/picrust2.smk"

if LONGITUDINAL == 'yes':
    include: 'rules/longitudinal.smk'
    print("Will perform a longitudinal analysis")
else:
    print("no longitudinal analysis")

The code above works only if I am running a longitudinal dataset. However, when I am not running the longitudinal analysis snakemake fails and says something like:

MissingInputException in line 70 of /mnt/c/Users/noahs/projects/tagseq-qiime2-snakemake-1/Snakefile:
Missing input files for rule all:

I think if I was able to add a similar conditional statement like the one I have for my external snakefile snakemake would not freak out about me not including the longitudinal snakefile.

Maarten-vd-Sande · Accepted Answer · 2020-11-23T07:20:03.760

4

You can define a list (or dict) of what you want as output outside of the rule all, and feed that to the input, something like this works:

myoutput = list()

if condition_1 == True:
    myoutput.append("file_1.txt")
if condition_2 == True:
    myoutput.append("file_2.txt")

rule all:
    input:
        myoutput

edit:

Either place myoutput as first in the input of rule all:

rule all:
    input:
        myoutput,
        raw_html = "raw_html_path",
        raw_zip = "raw_zip_path"

or make it named, and place it wherever:

rule all:
    input:
        raw_html = "raw_html_path",
        myoutput = myoutput,
        raw_zip = "raw_zip_path"

In Python (and snakemake) named positional arguments always go before named arguments.

edited Nov 23 '20 at 07:20

answered Nov 22 '20 at 08:22

Maarten-vd-Sande

3,413
10
27

Thank you for the response. I am not sure if I am making a mistake on my end because this structure is not working for me. Do all my outputs have to be appended lists for this to work? I currently have all my output files listed normally in my rule all and added ``myoutput`` below the rest of my input files and got this error: ```SyntaxError in line 178 of /mnt/c/Users/noahs/projects/tagseq-qiime2-snakemake-1/Snakefile: positional argument follows keyword argument``` 178 is the line in rule all where I added ``myoutput`` – Noah_Seagull Nov 22 '20 at 16:52
move the `myoutput` list to be the first in `all: input:`, or do sth like `all: input: optional = myoutput`. – Maarten-vd-Sande Nov 22 '20 at 19:44
Pinging you like this @Noah_Seagull , since I am not sure you get notified of my answer otherwise – Maarten-vd-Sande Nov 22 '20 at 20:47
This is still not working for me. It is possible that the append function does not like how my files are named, although my naming schema works fine in snakemake.Here is an example of the file names I am trying to pass through the append function as you suggested. ```myout.append("pw_diff = OUTPUTDIR + '/qiime2/asv/longitudinal/' + PROJ + ANALYSIS + 'pairwise-differences.qzv'")``` – Noah_Seagull Nov 23 '20 at 01:03
1

Thank you so much! you are a lifesaver. I was able to get it working by defining the positional arguments outside of rule all first. Since I needed to add one list of multiple inputs to another list of inputs I also had to use extend instead of append. But I would not have been able to figure it out with your help. – Noah_Seagull Nov 24 '20 at 16:25
@Noah_Seagull Can you please share your solution how you merged a list from expand statement with extend ? – Peter Pisher Sep 30 '21 at 16:28
Yes! The Solution is posted below. – Noah_Seagull Oct 02 '21 at 19:34

score 1 · Answer 2 · answered Oct 02 '21 at 19:33

Solution for merging list form expand statement:

I used a configuration file to pass the statements to the Snakefile

## Config.yml ##
# longitudinal analysis
perform_longitudinal: 'yes' # yes for longitudinal analysis

When 'yes' is entered in the configuration Snakemake will include additional variables in rule all and run an addition Snakefile to generate these files. There ended up being multiple Snakefiles so I used singularity to let Snakemake know that the rule all input files were for all 6 Snakefiles.

## Snakefile ##

configfile: "config.yaml"

LONGITUDINAL = config['perform_longitudinal']

 # rule all input files
 raw_html=file.txt, 
 raw_zip=file.txt,
 raw_multi_htmt=file.txt,
 raw_multi_stats=file.txt,
 Longitudinal_analaysis_files=file.txt

# rule all files excluding longitudinal analysis
rule_all_input_list=['raw_html','raw_zip','raw_multi_htmt','raw_multi_stats']

#longitudinal analysis files
rule_all_longitudinal_input=['Longitudinal_analaysis_files']

if LONGITUDINAL == 'yes':

    rule_all_input_list.extend(rule_all_longitudinal_input)

# conditionally add Snakefile to workflow
    include: 'rules/longitudinal.smk'

    print("Will perform a longitudinal analysis")

else:
    print("no longitudinal analysis")


rule all:
    input:
        data = rule_all_input_list

##### setup singularity #####

# this container defines the underlying OS for each job when using the workflow
# with --use-conda --use-singularity
singularity: "docker://continuumio/miniconda3"

##### load rules #####

include: "rules/qc.smk"
include: "rules/dada2.smk"
include: "rules/phylogeny.smk"
include: "rules/picrust2.smk"
include: "rules/differential.smk"

I have a less simplified version of how I got this working on GitHub https://github.com/nasiegel88/tagseq-qiime2-snakemake-1

Is it possible to add a conditional statement in snakemake's rule all?

2 Answers2