3

I'm running a snakemake pipeline that for a specific rule loads a container:

rule counts:
params:
    transcriptome=os.environ["INDEX"],
    outdir= (os.environ["OUTDIR"] + "/counts/"),
    indir= (os.environ["INDIR"] + "{sample}"),
    name = lambda wildcards: SAMPLES[wildcards.sample]
output:
    (os.environ["OUTDIR"] + "counts/" + "{sample}" + "/outs/web_summary.html")
container:
    "docker://marcusczi/cellranger_clean"
shell:
    """
    cellranger count --id={wildcards.sample} --transcriptome={params.transcriptome} --fastqs={params.indir} --sample={params.name}

    mkdir -p {params.outdir}
    mv ./{wildcards.sample}/ {params.outdir}

    """

Dry run looks fine, the rule itself I'm sure it works (tried it without the container). However, when I run it with docker I get this error:

Activating singularity image /some/path/.snakemake/singularity/c288fbc3fef5771f055a688c6678c24d.simg
/bin/sh: syntax error: unterminated quoted string
[    1.228141] reboot: Power down

And then it waits for the missing files, and fails.

I think the answer to this situation might be related to this previous question, but I have tried everything i can think of in terms of escaping characters (except for the wildcards and variables within curly brackets because I'm guessing it should be fine, and if not why am i even using snakemake :-( ). The paths for the directories I'm using are valid and exist, the name and wildcard "sample" are in the shape "sample_123", nothing fancy.

It's also worth saying that there are no single or double quotes in any of these variables.

Thank you!!

Software and OS: I am in macos catalina 10.15.5, running snakemake 5.20.1, and I have been using the beta version of singularity for macos (3.3.0-rc.1.658.g7427b73f1.dirty).

Running singularity outside Snakemake: I tried running the singularity outside snakemake, the software that I'm trying to run starts, but then complains that there is no disk left on space (which is not true). I'm running the singularity as sudo singularity run -B "$(pwd):$(pwd)" docker://marcusczi/cellranger_clean

I think this latest error might be either 1) I'm not running singularity as I should..? Or 2) A false statement of what is happening since cellranger (the software I'm trying to run) often has misleading error messages.

Minimal reproducible example: If you install snakemake, you should be able to reproduce my error when running snakemake -j1 --use-singularity in the same directory of the Snakefile.

Snakefile:

rule all:
    input:
        "output.txt"

rule counts:
    output:
        "output.txt"
    container:
        "docker://marcusczi/cellranger_clean"
    shell:
        """
        cellranger count --help
        echo "hurray!" > {output}
        """
compuTE
  • 120
  • 1
  • 10
  • 1
    This appears to be a singularity issue and not snakemake's. Note that what snakemake ultimately uses here is a singularity container and not docker container. Singularity will pull the image from docker hub and then convert it into a singularity image. While this conversion works perfectly fine most of the time, it is not fail-safe; conversion or the converted container may fail sometimes. My recommendation would be to test and ensure singularity container works fine before using it in snakemake pipeline. PS: Which OS, singularity and snakemake versions are you using? – Manavalan Gajapathy Jul 30 '20 at 14:03
  • Thank you @ManavalanGajapathy, I updated the question with the OS, softwares' versions and a new error I encountered when trying to run singularity outside snakemake. – compuTE Jul 31 '20 at 08:24
  • You may want to try to debug using `singularity shell` as well. As you noted, singularity for mac is still in beta, and in my limited testing, it didn't work well as expected. You may want to try its [conda version](https://anaconda.org/hcc/cellranger), though it appears to be unofficial. Tool's version available from dockerhub appears to be outdated, btw. – Manavalan Gajapathy Jul 31 '20 at 14:50
  • [Solution here](https://ubccr.freshdesk.com/support/solutions/articles/13000065620-singularity-build-error-no-space-left-on-device) might be of interest for the error `no disk left on space`. – Manavalan Gajapathy Jul 31 '20 at 16:27
  • Can you please include a reproducible example of how you are running your code? I have experience with singularity, but I have never used snakemake. – jkr Jul 31 '20 at 19:19
  • @jakub I updated the post with a minimal snakefile to reproduce my error (you will need to install snakemake). Thank you for checking this! – compuTE Aug 03 '20 at 12:00
  • The example worked for me on a linux machine with singularity 3.6.1. @compuTE - can you install the latest version of singularity? It's possible that it is an issue with singularity for mac. – jkr Aug 04 '20 at 15:16

0 Answers0