-1

When I'm trying to load classes dynamically using getattr, I get a module class instead of my real class.

module = importlib.import_module("bigpackage.animals")
class_ = getattr(module, "dog_input")
pprint(type(class_))
# Outputs <class 'module'>

My dog_input class:

from bigpackage.animals.animal import AbstractAnimal

class DogInput(AbstractAnimal):

    def __init__(self):
        self.b = "bb"

    @property
    def name(self):
        prefix = super(DogInput, self).name
        return prefix + "Name"

I have the following packages:

  • bigpackage (package)

    • animals (package)
      • abstract_animal (class)
      • dog_input (class)
      • init
    • init
    • server (class where I (try to) load the classes dynamically
J. Doe
  • 11
  • 1
  • 3
    `dog_input` is the file/module, within which there's a `DogInput` class… There may be multiple `class` definitions within `dog_input` too… – deceze Sep 11 '18 at 09:15
  • @deceze So what's the best solution? – J. Doe Sep 11 '18 at 09:16
  • 1
    @J.Doe: `dog_input.DogInput` would be the class. Or use `module.dog_input.DogInput`. Or import `bigpackage.animals.dog_input` and retrieve the `DogInput` attribute. – Martijn Pieters Sep 11 '18 at 09:17
  • `actual_class = getattr(class_, 'DogInput')`…? I don't know why this needs to go this route instead of `from bigpackage.animals.dog_input import DogInput` in the first place though. – deceze Sep 11 '18 at 09:17
  • 1
    @J.Doe: may I ask: why are you using `importlib.import_module()` here? What problem are you trying to solve? You don't seem to be that experienced with Python yet, this feels as if you are trying to run while still learning how to walk with Python. – Martijn Pieters Sep 11 '18 at 09:18
  • @deceze I have 'DogInput' dynamically, and it's not just this class, I have many others... – J. Doe Sep 11 '18 at 09:22
  • @MartijnPieters What's your suggestion? Just use __import__(name)? The import_module is a better practice for this case. What got the the feeling that I'm inexperienced? And In your first answer, what's the differences between dog_input.DogInput and the second choice of retreiving the DogInput attribute? – J. Doe Sep 11 '18 at 09:24
  • 2
    The real problem here is one of name resolution. Let's accept that you need to load this dynamically for whatever reason. That means you have a string at some point, say `"DogInput"`. That string needs to resolve to `dog_input.DogInput` somehow. Is your naming scheme up to this? Is there a reliable transformation from `dog_input` to `DogInput` or vice versa? Does that work for all other classes too? Or is there some other reliable mapping from your dynamic string to `module_name.ClassName`? – deceze Sep 11 '18 at 09:26
  • @deceze You're right, that's the problem. I can't map DogInput to dog_input. I can in first place name the python file `DogInput`, and then it will just be `DogInput.DogInput`, but I don't think that's a good practice. What's the solution for this kind of problem? I'm guessing I'm not the first one dealing with class reflection here. – J. Doe Sep 11 '18 at 09:31
  • @J.Doe: no, I wasn't making any suggestions about `__import__` here. I was trying to figure out why you need to use dynamic imports. deceze homed in on it better. – Martijn Pieters Sep 11 '18 at 09:35
  • There are certainly automatic ways to transform snake-case into camel-case. So you *could* use that as a resolution mechanism, *if* your class/module naming scheme is consistent (commonly known as *convention over configuration*). Then of course, Python prefers to be *explicit rather than implicit*, so perhaps you should just keep a `dict` of name mappings somewhere which you use to look up the module and class name from a string. – deceze Sep 11 '18 at 09:35
  • What do you mean by " Python prefers to be explicit rather than implicit"? And I get what you say, thank you. – J. Doe Sep 11 '18 at 09:38
  • 2
    https://www.python.org/dev/peps/pep-0020/#id3 – Use naming conventions: implicit. Use a dict to map names to modules: explicit. – deceze Sep 11 '18 at 09:42
  • @deceze But what does implicit and explicit mean in this context? – J. Doe Sep 11 '18 at 10:45
  • @J.Doe Explicit means that you explicitly specify where to fetch the class from - this will always work. Implicit means that you rely on an implicit rule to compute where the class is - this will break when someone does not adhere to your rule. – MisterMiyagi Sep 11 '18 at 11:24
  • As a side note: Python is not Java and neither impose nor even recommand the "one file per class" scheme. – bruno desthuilliers Sep 11 '18 at 11:41

1 Answers1

0

TLDR: You are only loading the module, not the contained class.

Note that packages and modules are mostly just arbitrary namespaces to Python. There is no strong mapping of a class to its contained module. Having a module dog_input implement the class DogInput does not make one the alias of another - DogInput is a regular member of dog_input, and the later may contain arbitrary other classes and values.

When you know where the class is located, the straightforward approach is to import the module, then fetch the class from it:

module = importlib.import_module("bigpackage.animals.dog_input")
class_ = getattr(module, "DogInput")
print(class_)

If you have only the class name but a consistent naming scheme, you can extract the module name from the class name. See this question on converting CamelCase to lower-case-with-underscores.

submodule_name = convert("DogInput")
module = importlib.import_module("bigpackage.animals.%s" % submodule_name)
class_ = getattr(module, "DogInput")
print(class_)

Note that this is generally frowned upon in Python. You implicitly rely on every maintainer knowing your naming convention - this can break easily, to say the least.

You can also have people provide a qualified name - dog_input.DogInput instead of just DogInput. Depending on how much nesting you allow (modules and/or content) this reaches from simple to very complex.

# relative module.class
qname = "dog_input.DogInput"
# add the module part to the import
module = importlib.import_module("bigpackage.animals.%s" % qname('.')[0])
# fetch the class part from the module
class_ = getattr(module, qname('.')[1])
print(class_)

This is a very powerful but also vulnerable approach. Use it if input comes from trusted, experienced users that need a lot of flexibility. Since it may allow executing arbitrary code, do not use it for public services.

If there is a fixed/known set of allowed classes, it is easiest to explicitly index them. This gives you a implementation flexibility as well as safety.

# bigpackage.animals
from dog_input import DogImport

# mapping from identifier to class
ANIMALS = {'DogImport': DogImport}

# bigpackage.server
from bigpackage.animals import ANIMALS
class_ = ANIMALS['DogImport']
print(class_)

This gives the most flexibility to you, but limits what users can do. It is ideal if code changes are required in the future, and users are external. Note that you may build the mapping dynamically, e.g. registering classes via decorators or entry_points.

MisterMiyagi
  • 44,374
  • 10
  • 104
  • 119