5

TL;DR: Is globals()[name] the CORRECT way to fall back to the "default"?

I have a large set of dynamically created classes that are defined from a YML file.

Dynamic class creation is accomplished via a combination of PyYAML yaml.safe_load_all and the dataclasses.make_dataclass (new in 3.7). I expect the specifications for these classes to occasionally change over time, which is why I chose YML as an easily understood format to describe them.

Python 3.7 is introducing new functionality (see PEP 562): a module-level __getattr__ function for managing module attribute access (there is also a module-level __dir__ function). It would be convenient to utilize this new function to allow importing of each dynamically created dataclass class from the module namespace, like so:

# some_module.py
from package_name import DataClassName1, DataClassName2

...and like so:

# package_name/__init__
from .my_dataclasses import DynamicDataClassesDict

def __getattr__(name):
    try:
        return DynamicDataClassesDict[name]
    except KeyError:
        # fall back on default module attribute lookup

In reading PEP 562 it isn't immediately clear to me how to fall back to default functionality for module attribute access. For a class, one would just call super().__getattr__(*args). I do see this line in one of the examples:

return globals()[f"_deprecated_{name}"]

This approach seems to work. Is globals()[name] the CORRECT way to fall back to the "default"? It doesn't seem to be, given that globals()[name] will raise a KeyError rather than the expected AttributeError.

Rick
  • 43,029
  • 15
  • 76
  • 119
  • I don't have Python 3.7 installed (yet), but if getting module attributes works like getting them from other objects, you should be able to use [`getattr()`](https://docs.python.org/3/library/functions.html#getattr) which supports an optional `default` argument which will be the return value if the attribute doesn't exist. As far as I can tell, PEP 562 doesn't discuss whether the new module `__getattr__()` will work with the built-in `getattr()` function, but I would expect that to be the case. – martineau Mar 21 '18 at 18:08
  • @martineau i haven't tried it yet but it's possible `getattr()` would work on modules that include a `__getattr__` function- but i wouldn't expect it to work on modules that do not, and it would also probably be blind to objects that the `__getattr__` isn't aware of. – Rick Mar 21 '18 at 19:17
  • Rick: If you're going to be using Python 3.7, can't you assume that all modules will have it (that work with that version)? If not there may be some way to test whether one does. I say all that because I think using `getattr()` would be the most "pythonic" ways to do what you want. – martineau Mar 21 '18 at 20:28
  • @martineau If I am reading the PEP correctly, it sounds to me that only modules that explicitly define `__getattr__` have it. I'm mostly trying to gain an understanding of how his new feature is meant to work; at this point I have my own code working to my satisfaction. – Rick Mar 21 '18 at 20:33
  • RIck: It's possible to make modules in earlier versions of Python also behave more like "normal" Python objects (specifically in this respect), which may be why a feature to make this easier was added to 3.7. Unfortunately counting on it being there can make code less useful if you want it to also be backwards compatible with earlier releases...unless you can devise a way that works in all of them (or all after certain point, say 2.6+) or start explicitly checking the version and doing things in more than one way, depending on which one is being used. – martineau Mar 21 '18 at 20:44
  • @martineau happily (for a variety of reasons) the only person who will ever set eyes on my code is me, so i have no problem living on the raggedy edge of python. i have seen some of the ways you refer to in the past but i'm excited about `__getattr__` as it seems far easier to use (though i can't remember what the other ways are). – Rick Mar 21 '18 at 20:53
  • 1
    Rick: There's answers showing how to do it in earlier versions under the question [`__getattr__` on a module](https://stackoverflow.com/questions/2447353/getattr-on-a-module). Personally I've always liked the one where the module in `sys.modules[]` is replace by an instance of a regular class. – martineau Mar 21 '18 at 21:06

1 Answers1

5

You have it backwards. The module-level __getattr__ function is only called as a last resort. There is nothing to fall back to - all other mechanisms have already failed to find an attribute of that name.

For example, if your module defines a global foo variable and someone accesses your_module.foo, then __getattr__ won't even be called.

The PEP explains this in the specification:

If an attribute is not found on a module object through the normal lookup (i.e. object.__getattribute__), then __getattr__ is searched in the module __dict__ before raising an AttributeError.

Hence, the correct "fallback behavior" is to raise an AttributeError.

Aran-Fey
  • 39,665
  • 11
  • 104
  • 149
  • 1
    makes sense. however, the default traceback message is lost when doing things this way. this makes me think it isn't *quite* the equivalent "default/fallback" I am looking for... – Rick Mar 21 '18 at 17:46
  • 2
    @RickTeachey: There are definitely ways in which that's inconvenient, but it's the way the `__getattr__` system works (and not just for module `__getattr__`). The original AttributeError and traceback are [unconditionally discarded](https://github.com/python/cpython/blob/v3.7.0b2/Objects/typeobject.c#L6445) before invoking `__getattr__`. (Module `__getattr__` goes through a [different code path](https://github.com/python/cpython/blob/v3.7.0b2/Objects/moduleobject.c#L679), but that code path doesn't let you use the default AttributeError either.) – user2357112 Mar 21 '18 at 17:53
  • 1
    @RickTeachey Hmm. Unfortunately I don't have python 3.7 yet, so I can't tell what difference it makes. However, the PEP is quite clear that _if_ `__getattr__` is called, its return value will be used as the attribute's value: _"If found, it is called with the attribute name and the result is returned."_ So raising an exception is really the only thing you can do if you don't want to return anything. – Aran-Fey Mar 21 '18 at 17:53
  • @user2357112 I suppose the only thing to be done, then is the same as in the case of any other object: re-create the traceback message yourself. – Rick Mar 21 '18 at 17:54