0

Question: how do I specify a conda environment with a distinct prefix in a snakemake rule without it crashing upon trying to deactivate the env?

background: I have conda environments stored in an alternate location (I made a given conda environment with conda create --prefix ./envname) than my baseline miniconda. I call these conda environments in their respective snakemake rules, as advised in the documentation.

rule hello: input: ... output: ... conda: "/path/to/conda/env" shell: "..."

I am unable to supply .yaml files to my Snakefile.

All my snakemake rules run beautifully. However, snakemake then tries to exit and does it using --name not --prefix

subprocess.CalledProcessError: Command 'conda env export --name '/path/to/env'' returned non-zero exit status     1.

Of course this will fail because conda environments referred to by 'name' can't have '/' in them.

So my snakemake crashes. My rules work great in the command line but I'm working with hundreds of samples so I need snakemake.

Also, I know people have said online that you can't do 'conda activate env' and that the workaround is to put a header saying 'source $HOME/miniconda3/etc/profile.d/conda.sh' which I have but this isn't fixing the issue because its within the Snakefile.

help! thank you!

wiscoYogi
  • 305
  • 2
  • 10
  • *"I am unable to supply .yaml files to my Snakefile."* Why is that? Like the documentation, I don't recommend users to use existing environments (it is prone to reproducibility failures). So, I'm interested to hear why YAMLs don't work - I'd be more interested to solve that problem. – merv Feb 02 '22 at 19:25
  • 1
    I was having space issues supplying a yaml file; the conda environment was being built right there and I had inadequate disk space. it'd be cool if one could specify to snakemake where they want the conda environments built. I changed my Snakefile run location and that fixed the issue since there was enough mem for the Conda env to be built tl;dr supplying conda environments (though it does run) is not a good idea and yaml files are appropriate though the caveat is that one must have enough disk space where the Snakefile is for the conda envs to be made – wiscoYogi Feb 03 '22 at 19:33
  • 1
    If you're on a typical HPC, you'll want to follow this: https://stackoverflow.com/a/67612296/570918. Namely, make sure the Conda package cache and default environment folders are not under your user home. Additionally, [Snakemake does have an argument](https://snakemake.readthedocs.io/en/stable/executing/cli.html#CONDA) to specify where to store environments. – merv Feb 03 '22 at 19:49
  • super helpful! thank you :-) – wiscoYogi Feb 03 '22 at 21:23

0 Answers0