3

I'm trying to setup my environment for a project but python isn't able to find the modules I've installed with pip.

I did the following:

mkdir helloTwitter
cd helloTwitter
virtualenv myenv
Installing setuptools, pip, wheel...done.
source myenv/bin/activate

pip install tweepy
Collecting tweepy
  Using cached tweepy-3.5.0-py2.py3-none-any.whl
Collecting six>=1.7.3 (from tweepy)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting requests>=2.4.3 (from tweepy)
  Using cached requests-2.11.1-py2.py3-none-any.whl
Collecting requests-oauthlib>=0.4.1 (from tweepy)
  Using cached requests_oauthlib-0.6.2-py2.py3-none-any.whl
Collecting oauthlib>=0.6.2 (from requests-oauthlib>=0.4.1->tweepy)
Installing collected packages: six, requests, oauthlib, requests-oauthlib, tweepy
Successfully installed oauthlib-2.0.0 requests-2.11.1 requests-oauthlib-0.6.2 six-1.10.0 tweepy-3.5.0

When I try to import the module it says it cannot be found.

The first entry in $PATH is helloTwitter/myenv/bin All the packages are showing up in the environments site-packages directory. I seem to be using the right python and pip. Which python outputs helloTwitter/myenv/bin/python Which pip outputs helloTwitter/myenv/bin/pip

Any advice on where I'm going wrong?

Marcus
  • 121
  • 2
  • 2
  • 7
  • 1
    Please give us the stack trace of the import error. – Zaur Nasibov Sep 22 '16 at 05:59
  • Could you tell me where I find the stack trace? – Marcus Sep 22 '16 at 08:51
  • >> When I try to import the module it says it cannot be found. - so how does it actually "say"? – Zaur Nasibov Sep 22 '16 at 09:13
  • In sublime I get an error saying the module cannot be found when I try to go to the definition. – Marcus Sep 22 '16 at 10:07
  • So, it is Sublime that has an issue, or you are getting an error when launching your program which has `import tweepy` in it? – Zaur Nasibov Sep 22 '16 at 12:11
  • Just tried it again. This is what I get: python heyTwitter.py Traceback (most recent call last): File "heyTwitter.py", line 1, in import tweepy ImportError: No module named tweepy – Marcus Sep 22 '16 at 12:32
  • Just got it to work...but not in the way I wanted. The site-packages directory for the virtualenv doesn't appear in sys.path but the user one does. I deactivated the environment, installed the module using pip with the user flag, then reactivated my environment and ran the program. So I guess my question is how do I configure python to look in the site packages directory in the virtual environment instead? – Marcus Sep 22 '16 at 12:39
  • Though helloTwitter/helloTwitterApp/myenv/bin appears first in my $PATH. I thought that should define where Python was looking first for packages? – Marcus Sep 22 '16 at 12:51
  • Are you actually activating the virtual environment before launching `python heyTwitter.py`? If not - then you are using the python which is installed in your system, and which uses system-wide installed libraries, not the virtualenv's. So, first make sure that virtual environment is activated and accept @W. Trombone's answer, if that was the issue :) – Zaur Nasibov Sep 22 '16 at 12:54
  • Yes the environment is active. – Marcus Sep 22 '16 at 12:56

4 Answers4

4

It looks like you're manually setting your $PATH to point to your virtual environment. The whole point of the myenv/bin/activate script is to take care of this for you.

Once you have activated your virtual environment, any package you install using pip will be placed in the relevant venv site-packages directory (in your case, myenv/lib/python2.7/site-packages). Things like pip --user are unnecessary when you are working in a virtual environment (assuming default behaviour). It's all automatic.

After running activate, you can check the python binary you are using with find -iname tweepy.

Aliases can cause issues too. which is an external command, and won't always pick these up. A type -a python will flush these out.

A quick test can be done by running helloTwitter/myenv/bin/python -c 'import tweepy' directly. If this behaves differently to however you are currently running python (i.e. doesn't throw an import exception), then this is your problem.

Hope that helps.

W. Trombone
  • 51
  • 2
  • type -a python python is /Users/Marcus/CodeProjects/helloTwitter/myenv/bin/python python is /usr/bin/python python is /usr/bin/python. I'm not sure what you mean by running python directly. I can execute a script in this directory. eg python helloTwitter.py >> prints success – Marcus Sep 22 '16 at 08:51
  • I don't get any exception or output when I try to run the import :S – Marcus Sep 22 '16 at 10:05
  • On Windows, I activated the virtual environment using the activate.bat in the venv\Scripts directory. After activating the environment, even though the path variable has been updated with the Scripts directory, when I run the 'python' command, it is still using the binary in the system global path. Have you come across this issue? – Sidharth Ramesh Aug 12 '22 at 10:16
3

Ok, I think I found a solution, if not an answer.

  1. I uninstalled the package and made sure it was not in system or user.
  2. I re-created the virtual environment.
  3. I checked that the environments python and pip were being used.
  4. This time when installing my package I added the --no-cache-dir option. The packages install successfully. Now Python can find the package.
derptop:environmentScience Marcus$ python 
>>> from tweepy import StreamListener 
>>> StreamListener 
<class tweepy.streaming.StreamListener'>

I checked the sys.path and it now includes the site-packages directory from the virtual environment, where previously it was absent.

Output of sys.path:

['', ....'/Users/Marcus/CodeProjects/environmentScience/myenv/lib/python2.7/site-packages']

As far as I can tell the sys.path was referencing the wrong site packages directory. Though I'm not sure why. I'm wondering if pips use of the cache was causing the site-packages reference to reset to system.

gary
  • 4,227
  • 3
  • 31
  • 58
Marcus
  • 121
  • 2
  • 2
  • 7
0

On Windows, if you are experiencing this issue, check that after activating your environment, the subsequent python commands are being run using the binary in the .venv\Scripts\Python executable.

I've seen this strange behaviour where even after the environment is activated using the activate.bat file in the virtual environment, AND the path to the Scripts\ directory is added to the system path, python commands still use the binary in the central Python installation in the system.

This leads to any packages being installed being installed NOT in the virtual environment, but in the global system location.

To work around this, when running pip or python commands after activating your environment, specify the correct path to the executable within your virtual environment, i.e.,

.venv\Scripts\python -m pip install setup.py

This should install the packages into the virtual environment and solve the issue.

Sidharth Ramesh
  • 646
  • 2
  • 6
  • 21
0

I solved this problem by change project directory. Move your project directory from desktop to another place. It helped me on Windows. I moved it from /Desktop to C:/some_folder

Radmir
  • 1
  • 1
    Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Nov 03 '22 at 13:12