I had some modules in the same path as my the script I'm running.
my_dir/
├── my_script.py
├── module_script_1.py
└── module_script_2.py
I had some code to import those modules on the fly when I needed them, and it worked like a charm:
print('Starting python module ' + name_value)
module = __import__(name_value)
df = getattr(module, name_value)(arg1 = val1)
print('Finished python module ' + name_value, df)
However, I'm trying to refactor this into a better design as the system is getting bigger.
I am attempting to use a namespace, so here's my folder structure now:
root_dir/
├── modules_dir/
| ├── __init__.py
│ ├── module_script_1.py
│ └── module_script_2.py
├── workflow_dir/
│ └── my_script.py
└── setup.py
Now, in the setup.py we have a namespace called root_dir:
setup(
name='root_dir',
namespace_packages=['root_dir'],
packages=[f'root_dir.{p}' for p in find_packages(where='root_dir')],
...
So that's my setup, but I can't seem to replicate my previous functionality of importing the modules on the fly as easily. Let me show you what I have in my_script.py now:
import root_dir.modules_dir # not necessary - just testing if import works
...
print('Attempting to import python module ' + name_value)
module = __import__('root_dir.modules_dir.' + name_value)
df = getattr(module, name_value)...
That gives an error:
df = getattr(module, name_value)...
AttributeError: module 'root_dir' has no attribute 'module_script_1'
Can you show me how to use namespaces correctly in this situation? It seems to get past the __import__ line but hasn't imported the correct thing.
how do I import from a module within a namespace on the fly?
