pytest failed to import local module (importing built-in module instead)

Question

I have the following Python 2.7 simplified project structure:

project/
  ├── libs/
  |     └── zipfile.py
  ├── tests/
  |     ├── __init__.py
  |     └── test_hello.py
  ├── hello.py
  └── main.py

I want this project to use the patched version of one of Python built-in modules (which in this example is zipfile) located in libs. Note that this is an external requirement, and I cannot change the project structure.

Below are the simplified implementation of each file:

libs/zipfile.py

def is_zipfile(filename):
    return "Patched zipfile called"

tests/test_hello.py

from hello import hello

def test_hello():
    assert hello() == "Patched zipfile called"

hello.py

import os
import sys

libs_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "libs"))
if libs_path not in sys.path:
    sys.path.insert(1, libs_path)

import zipfile

def hello():
    print(zipfile.__file__)  # to check which zipfile module is imported
    result = zipfile.is_zipfile("some_path")
    return result

main.py

from hello import hello

def main():
    print(hello())

if __name__ == "__main__":
    main()

When running the program directly (python main.py), I got the expected result:

/home/project/libs/zipfile.pyc
Patched zipfile called

However, when running pytest with project as the working directory (pytest -s), it failed:

/usr/lib/python2.7/zipfile.pyc
================================== FAILURES ===================================
_________________________________ test_hello __________________________________

    def test_hello():
>       assert hello() == "Patched zipfile called"
E       assert False == 'Patched zipfile called'
E        +  where False = hello()

tests/test_hello.py:4: AssertionError
========================== 1 failed in 0.13 seconds ===========================

I've tried a couple solutions presented in this SO post, such as running python -m pytest, but none has worked for me. Is there a way to successfully run this test in a non-hacky way?

IS this definitely the setup you have? When I copy this structure it fails with `pytest -s: tests/test_hello.py: ImportError: No module named hello` — match, Mar 08 '19 at 21:51
@match Woops, thank you for spotting it out, my tests directory actually has `__init__.py` in it. I'll update my question — Sam Tatasurya, Mar 08 '19 at 21:52

score 1 · Accepted Answer · answered Mar 09 '19 at 16:51

The reason your patched zipfile module is not being imported is it has already been imported before the test is started (likely by pytest or one of its dependencies)

I verified this by putting this at the top of hello.py:

if 'zipfile' in sys.modules:
    raise AssertionError('zipfile already imported')

I then get:

$ ./venv/bin/python -mpytest tests
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-4.3.0, py-1.8.0, pluggy-0.9.0
rootdir: /tmp/x, inifile:
collected 0 items / 1 errors                                                   

==================================== ERRORS ====================================
_____________________ ERROR collecting tests/test_hello.py _____________________
tests/test_hello.py:1: in <module>
    from hello import hello
hello.py:5: in <module>
    raise AssertionError('zipfile already imported')
E   AssertionError: zipfile already imported
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.14 seconds ============================

You could delete zipfile from sys.modules and then perhaps your copy would be the only one imported:

sys.modules.pop('zipfile', None)

That said, all of this seems like potentially a bad idea as anyone who has already imported that module will have access to the old zipfile and stubbing out stdlib implementation has high potential to break third party libraries which don't expect that.

You might have ~slightly better luck by patching out individual methods on the zipfile module directly (using something like mock.patch.object(zipfile, 'fn', ...)

Thank you for the detailed insight! Accepting this as the answer. I’ll think of a way to solve it based on your suggestion. — Sam Tatasurya, Mar 09 '19 at 17:40

pytest failed to import local module (importing built-in module instead)

1 Answers1