18

I've written a python program. And if I have a shebang like this one:

#!/usr/bin/python

and I make the file executable with:

$ chmod 755 program.py

I can run the program like so:

$ ./program.py

Here is the issue. I use the conda virtual environments. When I run the program like above, the system creates a subshell that does not recognize the active environment:

(my_env) $ ./program.py
ImportError: No module named pymongo

If I do it this way, however...

(my_env) $ python program.py
# blah blah... runs great

How do I specify the right environment for use in the subshell? Is it possible? I'd like to save my fingers the effort of typing the six character string that is python.

Another post, Shebangs in conda managed environments, briefly touches on this but does not provide the right answer. Instead of activating the environment in the subshell, it just says, go ahead and ignore the shebang... just use the $ python program.py syntax.

Community
  • 1
  • 1
meh
  • 2,591
  • 4
  • 20
  • 33

4 Answers4

29

In your script, change... 

#!/usr/bin/python

...to:

#!/usr/bin/env python

The python used by an activated conda environment is ${CONDA_PREFIX}/bin/python and not /usr/bin/python

Notice the difference?

(root) ~/condaexpts$ which python
/home/ubuntu/condaexpts/m3/bin/python

(root) ~/condaexpts$ /usr/bin/env python
Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:53:06) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

(root) ~/condaexpts$ source deactivate

~/condaexpts$ which python
/usr/bin/python

~/condaexpts$ /usr/bin/env python
Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
Nehal J Wani
  • 16,071
  • 3
  • 64
  • 89
  • 1
    Thanks @Nehal, that there is the right answer. Does this have a name? What do I search for to find this in the documentation? – meh Jan 31 '17 at 04:41
  • 1
    @meh It is a pretty common standard. Also go through this answer, which explains why we use this shebang. http://askubuntu.com/a/88314/95343 – Nehal J Wani Jan 31 '17 at 04:49
  • Good to know. Thanks! – meh Jan 31 '17 at 05:14
  • This depends on the parent conda environment already being activated, which was not my use case. – Dave X Sep 19 '17 at 19:05
  • Can I specify only the environment name? E.g. the path I get like this work only on my computer (as in `/home/ubuntu/`) even if the same environment is installed on the other one (but under a different absolute path). – nocibambi Jun 14 '19 at 13:08
20

conda run

If you always plan to run the script from a shell session where conda is defined, then another alternative is let Conda load the env using the conda run command. In this case, the shebang would be

#!/usr/bin/env conda run -n my_env python

The advantage here is that you don't need the env to be activated when you call ./program.py and you don't have to hardcode the location of the interpreter.

Note: This command was added as a "preview" in Conda v4.6.0 (see Release Notes) to address the issue of running a command inside an env.

merv
  • 67,214
  • 13
  • 180
  • 245
  • This looks like the best answer. If someone else can verify it works I will make this the accepted answer. I don't really use conda environments anymore and I'd have to set up an environment and a dummy project. Would take more time than I care to invest in this right now. – meh Oct 21 '19 at 15:51
  • @meh I can attest that it works for me on Mac OS X and Linux (shells configured with `conda init`). Not shebangs, but a Windows user confirmed that `conda.exe run python` worked for them [in this answer](https://stackoverflow.com/a/58458618/570918). – merv Oct 22 '19 at 17:04
  • 1
    Ok I'll give it to you. – meh Oct 28 '19 at 21:44
  • 2
    This gives me `/usr/bin/env: conda run -n my_env python: No such file or directory`. – 0x5453 Jun 09 '21 at 17:05
  • @0x5453 make sure `conda` is on your PATH (e.g., you have run `/path/to/miniconda3/condabin/conda init` for your shell). That's what "*run the script from a shell session where `conda` is defined*" means. – merv Jun 09 '21 at 18:15
  • 1
    @merv `conda` is on my PATH. Running `/usr/bin/env conda run -n my_env python -c 'import sys; print(sys.executable)'` in a shell gives me a sane result. It seems that my kernel is treating everything in my shebang after `/usr/bin/env` as a single argument. ([This](https://unix.stackexchange.com/q/399690/102168) question seems relevant.) – 0x5453 Jun 09 '21 at 21:09
  • @0x5453 oh - never hit that before. Thanks for the heads up! – merv Jun 10 '21 at 01:27
3

You can also point directly to the environment with the shebang line so you need not depend on something setting up the parent environment prior to calling the script.

First determine your preferred python environment's path:

$ . activate mypython
$ which python
/home/username/anaconda/envs/mypython/bin/python      # for example

Then use it in a script:

#!/home/username/anaconda/envs/mypython/bin/python
import os,sys
print sys.executable
print os.__file__

The above script would give output like this:

/home/username/anaconda/envs/mypython/bin/python
/home/username/anaconda/envs/mypython/lib/python2.7/os.pyc

Having a non-generic command in the shebang makes the script less portable, but if you depend the specific packages in a particular virtual environment, this is what you want.

Dave X
  • 4,831
  • 4
  • 31
  • 42
  • 2
    While I think this approach is usually fine, it should be noted that some packages use activation hooks to define the runtime context, and so using the interpreter directly without env activation would skip over this. – merv Oct 18 '19 at 16:50
2

The accepted answer didn't run for me, however this did:

#!conda run -n your_conda_env_name python

This requires conda in $PATH, and was on Windows.

Da1ne
  • 74
  • 3