0

I have a virtual environment my_env in which I installed Anaconda. When I type

which python

I get:

/user/pkgs/anaconda2/envs/my_env/bin/python

I have no errors importing numpy here:

(my_env) user@hostname:~/my_dir$ python
Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:42:40) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import numpy as np
>>>

But when I say 'import numpy as np' in a python program and run that from a shell script, I get:

(my_env) user@hostname:~/mydir$ ./program.sh 
Traceback (most recent call last):
  File "../python_program.py", line 3, in <module>
    import numpy as np
ImportError: No module named numpy

How can I fix this?

EDIT: I was asked what is in program.sh. The short answer is that I'm running different parameters in a loop. The long answer is:

#/bin/bash

i=0
while read a1 b1 c1 d1 e1 f1 g1 h1 i1
    do
    i=$(($i+1))
    mkdir RUN_EXP$i
    cp $a1 RUN_EXP$i
    cd RUN_EXP$i
    ../python_program.py --filename $a1 --reps $b1 --pop $c1 --susc $d1 --exp_trans $e1 --inf_period $f1\ $g1 --eps $h1\ $i1
    cd ..
    done < readparas.txt

The file readparas.txt has lines containing filename, reps, pop, susc, exp_trans, inf_period, and eps like this:

run_1.txt 50 162 0.30 0.1 5 9 0.1 0.25
run_1.txt 50 162 0.30 0.3 5 9 0.1 0.25
...
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
StatsSorceress
  • 3,019
  • 7
  • 41
  • 82

1 Answers1

2

Your shell script doesn't care about having a virtualenv active (it starts in a clean environment).

Instead of ../python_program.py you need to have the full executable path

 export PYTHON_ENV=/user/pkgs/anaconda2/envs/my_env

 $PYTHON_ENV/bin/python ../python_program.py --filename $a1 ...

Or you can append this to the top of python_program.py

#!/usr/bin/env python

Refer: The importance of env (and how it works with virtualenv)

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • Why is the full executable path necessary? It wasn't with a similar program I ran before. – StatsSorceress Mar 05 '17 at 22:55
  • Because you are using a `virtualenv` where `numpy` is installed at. Your other program was using your system Python and had all the modules available. – OneCricketeer Mar 06 '17 at 03:20
  • Except the problem is also happening outside the virtualenv. – StatsSorceress Mar 08 '17 at 20:19
  • Yes, the error happens outside the virtualenv because you don't have `numpy` there. You need to tell your script which `python` executable needs to run your code. – OneCricketeer Mar 08 '17 at 20:29
  • 1
    @StatsSorceress To clarify: your shell script starts up a new, cleaner, shell environment. It does *not* inherit all PATH settings from the standard shell you issue your `python` command in, hence you need to specifically use that path to the virtualenv Python –  Mar 08 '17 at 20:37
  • @StatsSorceress You sure about that? Otherwise, you would have no `ImportError`. You might have it installed from Python2, but not Python3, or you aren't using Anaconda libraries as your `python` that is executed on your `PATH`. – OneCricketeer Mar 08 '17 at 20:49
  • @cricket_007 Thank you, I completely missed the addition of 'env'. I apologize for the misinterpretation! I've accepted your answer. – StatsSorceress Mar 08 '17 at 21:18
  • I just recently added it – OneCricketeer Mar 08 '17 at 21:20