Using libraries that use patched and non-patched version of another library

Question

I guess this question (Import not Working) means the following and it is kind of interesting:

Library A and B are actually grequest and multiprocessing. (I heard) grequest patches socket but multiprocessing can't use the patched version.

I want to use two libraries A and B. The problem is that library A internally imports patched version of library C, but library B internally imports the non-patched version of library C.

libraryA.py:

import numpy as np
def f():
    print("patched")
np.array = f

libraryB.py:

import numpy as np
def g():
    return np.array([1,2,3])

my_program.py:

import libraryA
import libraryB
libraryB.g()

result (python3 my_program.py):

Traceback (most recent call last):
  File "my_program.py", line 3, in <module>
    libraryB.g()
  File ".../test/libraryB.py", line 3, in g
    return np.array([1,2,3])
TypeError: f() takes 0 positional arguments but 1 was given

Problem:

libraryB should use the non-patched version of numpy, but it is using the patched version, and so libraryB.g() in my_program.py breaks. How to fix -this- ?

There are no two separate versions. Modules are singletons in Python, they are loaded **once** and all an `import` statement does is bind names. There is only the patched 'version' available. — Martijn Pieters, Jul 20 '17 at 06:46
The only way around this is to patch the patching library to stop it from patching directly and find a different way of making that library work. — Martijn Pieters, Jul 20 '17 at 06:47
yea. so the only way is to rewrite library A to make it inherit from numpy and just patch the inherited class? That sounds fun and interesting... — hamster on wheels, Jul 20 '17 at 06:47
It may be possible to patch library A, but that depends heavily on exactly what they do to numpy and how they use the patched version. — Martijn Pieters, Jul 20 '17 at 06:49

Martijn Pieters · Accepted Answer · 2017-07-20T06:56:57.323

There are no two separate versions. Modules are singletons in Python, they are loaded once and all an import statement does is bind names (with the first such statement triggering the loading). There is only the patched 'version' available.

The only way around this is to patch or replace the patching library to stop it from patching directly and find a different way of making that library work. It depends heavily on the patching library how this could be achieved.

This is one reason grequest remains a very specialised usecase; it patches the standard library, making a large number of other code that relies on the standard library to work in a certain way to be incompatible. If you were looking to combine grequests and multiprocessing you'd more likely have to find alternatives for one or the other approach.

Using libraries that use patched and non-patched version of another library

1 Answers1