25

Why python is not finding boto3 when pip or pip3 shows it's installed.

I'm on Mac machine. Tried pip/pip3 with / without sudo. Tried changing the PATH variable as well with few paths but no luck.

PATH: /Library/Frameworks/Python.framework/Versions/3.5/bin:/Users/arun/.sdkman/candidates/gradle/current/bin:/Users/arun/aks/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

and

$ python -c 'import sys; print sys.path'|tr ',' '\012'
[''
 '/Library/Python/2.7/site-packages/RBTools-0.5.7-py2.7.egg'
 '/Library/Python/2.7/site-packages/wfawsclitools-0.1-py2.7.egg'
 '/Library/Python/2.7/site-packages/PyYAML-3.12-py2.7-macosx-10.12-intel.egg'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload'
 '/Library/Python/2.7/site-packages'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python'
 '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC']

and

$ python -m site
sys.path = [
    '/Users/arun/ubuntu-vagrant-box',
    '/Library/Python/2.7/site-packages/RBTools-0.5.7-py2.7.egg',
    '/Library/Python/2.7/site-packages/wfawsclitools-0.1-py2.7.egg',
    '/Library/Python/2.7/site-packages/PyYAML-3.12-py2.7-macosx-10.12-intel.egg',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
    '/Library/Python/2.7/site-packages',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC',
]
USER_BASE: '/Users/arun/Library/Python/2.7' (doesn't exist)
USER_SITE: '/Users/arun/Library/Python/2.7/lib/python/site-packages' (doesn't exist)
ENABLE_USER_SITE: True

Steps I ran so far:

[arun@ip-10.11.33-65 ~/aks] $ pip install --upgrade pip
Requirement already up-to-date: pip in /usr/local/lib/python2.7/site-packages
[arun@ip-10.11.33-65 ~/aks] $ pip install boto3
Requirement already satisfied: boto3 in /usr/local/lib/python2.7/site-packages
Requirement already satisfied: botocore<1.6.0,>=1.5.0 in /usr/local/lib/python2.7/site-packages (from boto3)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python2.7/site-packages (from boto3)
Requirement already satisfied: s3transfer<0.2.0,>=0.1.10 in /usr/local/lib/python2.7/site-packages (from boto3)
Requirement already satisfied: docutils>=0.10 in /usr/local/lib/python2.7/site-packages (from botocore<1.6.0,>=1.5.0->boto3)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python2.7/site-packages (from botocore<1.6.0,>=1.5.0->boto3)
Requirement already satisfied: futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" in /usr/local/lib/python2.7/site-packages (from s3transfer<0.2.0,>=0.1.10->boto3)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python2.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.6.0,>=1.5.0->boto3)
[arun@ip-10.11.33-65 ~/aks] $ pip3 install boto3
Requirement already satisfied (use --upgrade to upgrade): boto3 in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages
Requirement already satisfied (use --upgrade to upgrade): s3transfer<0.2.0,>=0.1.0 in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages (from boto3)
Requirement already satisfied (use --upgrade to upgrade): botocore<1.5.0,>=1.4.1 in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages (from boto3)
Requirement already satisfied (use --upgrade to upgrade): jmespath<1.0.0,>=0.7.1 in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages (from boto3)
Requirement already satisfied (use --upgrade to upgrade): docutils>=0.10 in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages (from botocore<1.5.0,>=1.4.1->boto3)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil<3.0.0,>=2.1 in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages (from botocore<1.5.0,>=1.4.1->boto3)
Requirement already satisfied (use --upgrade to upgrade): six>=1.5 in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.5.0,>=1.4.1->boto3)
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[arun@ip-10.11.33-65 ~/aks] $ which python
/usr/bin/python
[arun@ip-10.11.33-65 ~/aks] $ `which python` --version
Python 2.7.10
[arun@ip-10.11.33-65 ~/aks] $ python
Python 2.7.10 (default, Jul 30 2016, 19:40:32) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named boto3
>>> ^D
[arun@ip-10.11.33-65 ~/aks] $

On the other side, if I run python3, it works for import boto3 but not import boto.

AKS
  • 16,482
  • 43
  • 166
  • 258
  • What is the output of `python -c 'import sys; print sys.path'`? Does it contain `/usr/local/lib/python2.7/site-packages`? – Leon Mar 01 '17 at 19:12
  • @Leon No it doesn't, updated my question with that output for your ref. – AKS Mar 01 '17 at 19:16
  • Try this: `PYTHONPATH=/usr/local/lib/python2.7/site-packages python -c 'import boto3; print dir(boto3)'`. – Leon Mar 01 '17 at 19:19
  • Please include the output of `python -m site` in your question. – wim Mar 01 '17 at 19:23
  • It seems that your `pip` and `python` executables (for version 2.7 of python) do not agree on the package installation/search path. – Leon Mar 01 '17 at 19:24
  • @wim hm that tells something there. Included the output and updated my post. – AKS Mar 01 '17 at 19:37

4 Answers4

75

Install like this:

python -m pip install --user boto3
wim
  • 338,267
  • 99
  • 616
  • 750
  • Worked, I wonder why you got 89.9k :) Also, for python3 prompt, `import boto` was not working, running `python3 -m pip install --user boto` fixed that too. So, now my `python` and `python3` both have `import boto` and `import boto3` working in their prompts or in a .py python program where `import` statement is present for boto/boto3. – AKS Mar 01 '17 at 21:27
  • 1
    Had the same issue with `import credstash`, fixed by running `python -m pip install --user credstash` – AKS Mar 01 '17 at 22:44
  • Helped me too much. Thanks! – Amit Teli Aug 01 '17 at 17:08
  • 2
    You... Are... A... GOD – Kyle Bridenstine May 17 '18 at 20:57
  • Although the answer is almost year old it helped me today a lot. Btw, do you know why this solutions works and the usual way `pip3 install module` doesn't? – Nav Mar 27 '19 at 18:33
  • what does `user` represent? the login to the computer? – ziggy Mar 28 '19 at 13:35
  • I get `/usr/local/lib/python2.7.9/bin/python: No module named pip` after running this command – Victor Cui Jul 05 '19 at 20:36
  • @Nav messed up pip scripts hanging around in $PATH. I describe the root causes and how to untangle the problem in [this answer](https://stackoverflow.com/a/58665743/674039). – wim Nov 11 '19 at 19:13
  • as of 2021 using python 3.6.5 that answer still works – rightsized May 14 '21 at 13:40
10

I ran into this same issue the other day. I had python 2.7 installed on my Mac (there by default I presume). I then installed homebrew and used it to get the latest version of python setup:
brew install python

Then, I had two versions of python, which is okay. Brew installed pip3 along with Python 3.6.5.

So then, I was able to run this to get boto3: pip3 install boto3 To ensure that it is accessible, use the python3 command, not the older python. So in my case, I ran my code with:
python3 signURL.py

Michael
  • 101
  • 1
  • 4
2

Just in case this worked for me after upgrading to ansible 2.4.0.0:

pip install --upgrade --user boto3

That recreated my $HOME/Library/Python/2.7/lib/python/site-packages

nbari
  • 25,603
  • 10
  • 76
  • 131
-1

so I'm an uber noob at python, doing something for work but posting just in case this helps someone... I had the same error and all it was was that I had everything downloaded under python3 and I was typing the python some_script.py causing the:

File "some_script.py", line 4, in <module>
    import boto3

changing it to python3 some_script.py fixed it/found the proper module

jawn
  • 851
  • 7
  • 10