Dynamic output in snakemake

Question

I'm using snakemake to develop a pipeline. I'm trying to create symbolic links for every file in a directory to a new target. I don't know ahead of time how many files there will be, so I'm trying to use dynamic output.

rule source:
    output: dynamic('{n}.txt')
    run:
        source_dir = config["windows"]
        source = os.listdir(source_dir)
        for w in source:
            shell("ln -s %s/%s source/%s" % (source_dir, w, w))

This is the error I get:

WorkflowError: "Target rules may not contain wildcards. Please specify concrete files or a rule without wildcards."

What is the issue?

I have never tried `dynamic`, but the examples given in the documentation have a different way of using this than what you do: http://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#dynamic-files. In particular, there is a driving `all` rule, that have the dynamic stuff as its input. Dou you have such a rule ? — bli, Jun 30 '17 at 09:05

score 6 · Accepted Answer · edited Aug 20 '20 at 20:44

In order to use the dynamic function, you need to have another rule in which the dynamic files are the input. Like this:

rule target:
  input: dynamic('{n}.txt')
  
rule source:
  output: dynamic('{n}.txt')
  run:
    source_dir = config["windows"]
    source = os.listdir(source_dir)
    for w in source:
      shell("ln -s %s/%s source/%s" % (source_dir, w, w))

This way, Snakemake will know what it needs to attribute for the wildcard.

Hint: when you use a wildcard, you always have to define it. In this example, calling dynamic in the input of the target rule will define the wildcard '{n}'.

Dynamic output in snakemake

1 Answers1

Linked