I am using Python 3.8.1, and since 3.7, we can use __getattr__ on modules to import import undeclared attributes. According to PEP 562 (https://www.python.org/dev/peps/pep-0562/):
The __getattr__ function at the module level should accept one argument which is the name of an attribute and return the computed value or raise an AttributeError
After reading the PEP and this related question (__getattr__ on a module), there are two observed behavior that I do not understand. Any help is welcome here!
1. The __getattr__ method is called even when importing an existing attribute
This is particularily weird, especially since, according to the PEP:
If an attribute is not found on a module object through the normal lookup (i.e. object.__getattribute__), then __getattr__ is searched in the module __dict__ before raising an AttributeError. If found, it is called with the attribute name and the result is returned. Looking up a name as a module global will bypass module __getattr__. This is intentional, otherwise calling __getattr__ for builtins will significantly harm performance.
The performance implications of this PEP are minimal, since __getattr__ is called only for missing attributes.
# module1.py
foo = 0
def __getattr__(name):
print('__getattr__, name: ' + name)
if name == 'foo':
res = 1
elif name == 'bar':
res = 2
else:
raise AttributeError
return res
>>> from module1 import foo
__getattr__, name: __path__
>>> foo
0
>>> from module1 import bar
__getattr__, name: __path__
__getattr__, name: bar
>>> bar
2
>>> from module1 import baz
__getattr__, name: __path__
__getattr__, name: baz
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'baz' from 'module1' (C:\Users\bobicph\Projects\Python\Test\module1.py)
>>> baz
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'baz' is not defined
Which means that __getattr__ is called at least twice to import a missing attribute, and...
2. The __getattr__ method is called a third time if it doesn't raise an AttributeError
It's not much, but it doesn't say in the PEP that it must throw an AttributeError, and importing the attribute twice is a bother if the import is long for example.
# module2.py
import time
foo = 0
def f(name):
time.sleep(10)
return 2
def __getattr__(name):
print('__getattr__, name: ' + name)
if name == 'foo':
res = 1
elif name == 'bar':
res = f(name)
else:
res = 3
return res
>>> from module2 import foo
__getattr__, name: __path__
>>> foo
0
>>> from module2 import bar # Takes 20 seconds
__getattr__, name: __path__
__getattr__, name: bar
__getattr__, name: bar
>>> bar
2
>>> from module2 import baz
__getattr__, name: __path__
__getattr__, name: baz
__getattr__, name: baz
>>> baz
3
>>> from module2 import bar # Takes another 20 seconds
__getattr__, name: __path__
__getattr__, name: bar
__getattr__, name: bar
>>> bar
2
Am I missing something (especially for 1.)? Are those behavior meant to be? If so, how come?
