I have a folder structure like below (attached image)
In Main.py I imported all the folders. Now Main.py calls subMain.py which is inside the Project 1. subMain.py imports spark, common etc. Now utils.py has functions from helper.py hence need to import helper.py
This throws module import error that "No module named Common"
I tried putting init.py in all the folders, however no luck. Can anyone help please? Thanks in advance.
Looks like what you want to do is a relative import of helper.py from utils.py
with the following directory structure
.
├── main.py
└── project
├── __init__.py
├── common
│ ├── __init__.py
│ ├── helper.py
│ └── queries.py
├── spark
│ ├── __init__.py
│ └── utils.py
└── submain.py
in your utils.py , you should be able to import commons using
from ..common import helper
To learn more about relative imports, see this write-up and this SO question. I am using python 3.6
I guess that by multiple subfolder you mean nested subfolder.
There are several ways, depending on whether you want to create a pip installable library (not necessarily available via pypi), or if you want to have a bunch of files bundled together.
Please look into the structure of an existing python project on github, and try to reproduce it. Pay attention to where the __init__.py files are added, and look under the setup.py if there is any.
This repo https://github.com/beurtschipper/Depix can be an example if you do not wish to create a pip installable library (no setup.py file is present).
Otherwise this one https://github.com/nipy/nilabels can be of help if you want to create a library (setup.py file is present).
Without the __init__.py and from python version > 3, you can access python files via relative import, but this solution is discouraged, as the user may start running the code from different folder (not necessarily the one where you have the main.py, breaking the relative paths).
An alternative is to use the pythonpath (but this is not a solution I would recommend. If you have multiple projects it can get very busy, or you may need hacks at module level to add the relative path to the python path programmatically)
https://docs.python.org/3/using/cmdline.html#environment-variables
