1

I have a simple test script:

import requests
response = requests.get('http://httpbin.org/get')
print response.text

It works when the python script is named test.py but fails if named email.py or logging.py:

Traceback (most recent call last):
  File "./email.py", line 3, in <module>
    import requests
  File "/usr/lib/python2.7/dist-packages/requests/__init__.py", line 53, in <module>
    from urllib3.contrib import pyopenssl
  File "/usr/lib/python2.7/dist-packages/urllib3/__init__.py", line 16, in <module>
    from .connectionpool import (
  File "/usr/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 59, in <module>
    from .request import RequestMethods
  File "/usr/lib/python2.7/dist-packages/urllib3/request.py", line 12, in <module>
    from .filepost import encode_multipart_formdata
  File "/usr/lib/python2.7/dist-packages/urllib3/filepost.py", line 15, in <module>
    from .fields import RequestField
  File "/usr/lib/python2.7/dist-packages/urllib3/fields.py", line 7, in <module>
    import email.utils
  File "/home/ubuntu/temp/email.py", line 4, in <module>
    response = requests.get('http://httpbin.org/get')
AttributeError: 'module' object has no attribute 'get'

It appears that requests imports urllib3, which imports email built-in module. Why is Python not finding the built-in email module first, instead of looking in the current path for email.py?

Is there a way to make this work, or do I just have to always avoid naming my Python scripts any built-in module that may be imported by any dependency?

wisbucky
  • 33,218
  • 10
  • 150
  • 101
  • See also ["Force import module from Python standard library instead of PYTHONPATH default"](http://stackoverflow.com/q/2952045/12892) and ["How to access a standard-library module in Python when there is a local module with the same name?"](http://stackoverflow.com/q/1900189/12892). – Cristian Ciupitu Feb 23 '15 at 23:31

4 Answers4

5

First, as you indicated in a comment, Python checks for a 'built-in' module of that name. Not all modules in the standard library are 'built-in'. You can see the list by:

print sys.builtin_module_names

If it's not found there, the order searched is outlined by Burhan's accepted answer here:

What is the extent of the import statement in Python

Python searches for things it can import in the following order:

  • From the directory where the script was executed.
  • From the directories in the PYTHONPATH environment variable (if its set).
  • From the system-wide Python installation's module directory.

In your case, email is not built-in, so the current directory is checked first.

So, yes, don't shadow a python library name. Technically you could shadow a built-in module's name, but please - please don't. It makes Python cry.

Community
  • 1
  • 1
robert_x44
  • 9,224
  • 1
  • 32
  • 37
  • Ah, I thought "built-in" was the same as "standard" modules, but it's not. There are actually very few "built-in" modules. – wisbucky Feb 23 '15 at 23:45
2

Take a look at sys.path and you'll see '' as the first entry:

>>> import sys
>>> sys.path
['',
 '/.../3.3/lib/python33.zip',
 '/.../3.3/lib/python3.3',
 '/.../3.3/lib/python3.3/plat-darwin',
 '/.../3.3/lib/python3.3/lib-dynload',
 '/.../3.3/lib/python3.3/site-packages']

That '' is the current directory.

You could modify sys.path, but it's a wiser decision, long-term, to just not give python files names that match builtin modules.

Benjamin Riggs
  • 648
  • 1
  • 7
  • 13
0

That's not a problem with Requests.

The "problem" comes from Python. email and logging are standard modules, it is not advisable to name your own modules same unless you understand and are willing to resolve conflicts.

Dima Tisnek
  • 11,241
  • 4
  • 68
  • 120
Turtled
  • 9
  • Why not? According to https://docs.python.org/2/tutorial/modules.html#the-module-search-path , python first searches for built-in modules before looking in directories for .py – wisbucky Feb 23 '15 at 23:06
  • I understand now. https://docs.python.org/2/py-modindex.html is a list of standard modules, the most of which are not "built-in modules". – wisbucky Feb 23 '15 at 23:48
0

The reason Python is looking for the modules in the current directory first is that that's how Python works. If you look at sys.path you will see that its first element is almost always '' which indicates the current directory. This applies to everything but the built-in modules, which are "imported" at interpreter startup. (They are not really imported, just assigned into sys.modules.)

In general, you are supposed to not name your own modules the same names as other modules you're using (whether they came with Python or not). I tend to put my initials on scripts I'm working with just to avoid this pitfall.

Of course, you can just manipulate sys.path so it doesn't look for modules in the current directory:

import sys
if not sys.path[0]:
   del sys.path[0]
import requests

Note that you only need to remove the '' entry when you import. You should put it back after you do your imports, in case you need to import modules of the same name from your own script's directory. A context manager is handy for this.

import sys
from contextlib import contextmanager

@contextmanager
def no_cwd_imports():
    old_path = sys.path[:]
    if not sys.path[0]:
        del sys.path[0]
    try:
        yield
    finally:
        sys.path[:] = old_path

with no_cwd_imports():
    import requests
kindall
  • 178,883
  • 35
  • 278
  • 309