2

imagine using a python library. Suppose that this library has one script and a couple of classes that are of interest. All these classes are defined in individual files. Suppose that class D is imported and used by class A,B and C in multiple ways. In the same way class A uses class B and class B uses class C. The script makes direct use of class A and thus direct and indirect use of class D. Now suppose that you want to customize D and locally define a class D' that is derived from class D.

What would be the most elegant way to make the script and the other classes (A,B, C) use class D' instead of D?

Thank you and have a nice day!

DisplayName
  • 67
  • 4
  • 1
    There is not general answer, what are you attempting to do specifically? – Olivier Melançon Mar 04 '19 at 18:25
  • 1
    This is totally going to depend on *how exactly* the imports are managed. For example, this will be annoyingly difficult if the scripts use `from d import D` instead of `import d` and then in the files use `d.D` to reference `D`. – juanpa.arrivillaga Mar 04 '19 at 18:26
  • For common libraries such as `json` you can inherit the classes and then use that class in function calls, e.g. `json.dump(x, fp, encoder=YourNewClass)` – Rocky Li Mar 04 '19 at 18:30

1 Answers1

1

You can monkey patch the library.

# a.py
import d
class A():
    my_d = d.D()

# script.py
A.d.D = D_prime

def foo():
    a_cls = a.A() # uses A.d.D is D'

The way that a.py imports D matters

# a.py
from d import D
class A():
    my_d = D()

# script.py
A.D = D_prime

def foo():
    a_cls = a.A() # uses A.D is D'

Other import schemes may involve similar patterns. Something like this might be very difficult to patch.

def builder():
    from d import D
    return D()

It may also be helpful to see how the mock library does this. If it's a toy, you could use this directly. patch in particular is interesting.

@patch('A.d.D', new=D_prime)
def my_func(...):
    a = A() # a.d.D is D_prime

Monkey patching is a code smell, depending on unittest in application code is a smell. None of it is "elegant". If you are the library author, please support dependency injection instead.

If you're brave, you might pull out some of the patch decorator into something that's not focused on mocks. Because this pattern exists in the unittest library, it can be considered pythonic and "elegant" with the caveat above.

If you're interested in how this works, you're modifying the symbol table for the module a. See globals and locals for details.

munk
  • 12,340
  • 8
  • 51
  • 71
  • Thank you (and everyone else). Once I got my head around it, it turned out to be quite easy. For anyone else having problems to understand it, here is a less abstract example that I found once I knew the word monkey patch: (https://stackoverflow.com/questions/19545982/monkey-patching-a-class-in-another-module-in-python) . – DisplayName Mar 05 '19 at 10:25
  • Follow up: why does my monkey patch break down? The lib I'm using is ray 0.6.2 (https://github.com/ray-project/ray) and I succesfully applied my patch like this: `ray.tune.trial.Trial.init_logger = custom_init_logger`. So far, so good. Now I want to also patch another function `ray.rllib.evaluation.metrics.summarize_episodes = custom_summarize_episodes`, which doesnt work. Working back I found that while `ray.rllib.agents.dqn.DQNAgent.train = exit` works, `ray.tune.trainable.Trainable.train = exit ` doesnt. 'Trainable' is imported and it's train function called in the agents train function. – DisplayName Mar 07 '19 at 10:06
  • @DisplayName I suspect what's happening is `ray.rllib.agents.dqn.DQNAgent.train` is imported from `ray.tune.trainable.Trainable.train`. If the patch happens after this import, and you've patched `ray.tune.trainable.Trainable.train`, the place where it's used has a reference to the original function. – munk Mar 07 '19 at 15:10