Python: Importing pandas into main and modules?

Question

I have a python script with a structure that looks like the following:

/
  __init__.py
  main.py
  /modules
    fbm.py

I am attempting to split some functions out from main.py into fbm.py and then import fbm.py as a module using.

sys.path.extend([f'{item[0]}' for item in os.walk("./app/modules/") if os.path.isdir(item[0])])
import fbm

This is working.

What I am having trouble with is where I need to call pandas (or another import) from a module. I'm currently importing pandas in main.py with import pandas as pd. When I call the function from fbm an error is thrown as soon as it hits a reference to pd stating that NameError: name 'pd' is not defined. pd is defined in main.py, but not fbm.py. It thus works in main, but not in fbm.

So, my question is: Is it good and appropriate to import pandas as pd in both the main.py and each module which requires pandas?

Will this have an impact on memory usage, ie loading multiple copies of pandas
is there a better way to handle this?

You need to import `pandas` in all files which use it, there is no other way. — Maurice Meyer, Apr 19 '23 at 23:13
Modules can't reference variables declared in main. If a module needs pandas, it needs to import it itself. — Barmar, Apr 19 '23 at 23:13
There's little impact on memory. Pandas is only loaded once, it's just added to the module's namespace. — Barmar, Apr 19 '23 at 23:14
Does this answer your question? [Does python optimize modules when they are imported multiple times?](https://stackoverflow.com/questions/296036/does-python-optimize-modules-when-they-are-imported-multiple-times) — ti7, Apr 19 '23 at 23:15

score 2 · Accepted Answer · answered Apr 19 '23 at 23:21

It is appropriate to import pandas (or any other library) in both the main script and each module that requires it. This is a common practice and ensures that each module can function independently, which makes the code more modular and easier to maintain.

Regarding your concerns about memory usage: Python is smart enough to handle imports efficiently. When a module is imported for the first time, it's loaded and cached in the sys.modules dictionary. So, if the same module is imported again (even with a different alias), Python will use the cached module instead of loading it again and again. So, there won't be any significant impact on memory usage.

Python: Importing pandas into main and modules?

1 Answers1