1

I have a python project with a folders and file structure like:

root directory
    classes
        my_common_class.py
        my_class_1.py
        my_class_2.py
    my_virtual_env
        bla bla bla
    my_wonderful_script.py

Both my_class_1.py & my_class_2.py are used on my_wonderful_script.py and both of them inherit from my_common_class.py.

On each class I have something like:

try:
    from classes.my_common_class import myCommonClass
except ImportError:
    from my_common_class import myCommonClass

This approach feels a bit ugly. This is because on each of those classes I test each method executing each class file:

if __name__ == '__main__':
    #My quick methods testing goes here
    ....
    ....

When I activate the virtual enviroment on my local machine, and all tests and my_wonderful_script.py run smoothly. When I do the same on my server, I get an exception:

  File "<path>/my_wonderful_script.py", line 5, in <module>
    from classes.my_class_1 import myClass1
  File "<path>//classes/my_class_1.py", line 10, in <module>
    from my_common_class import myCommonClass
ModuleNotFoundError: No module named 'my_common_class'

I assume I am missing something in the way I am handling the import on each machine.

Is there a "prettier" way to do this?

Javi M
  • 97
  • 9
  • 3
    Does this answer your question? [Relative imports for the billionth time](https://stackoverflow.com/questions/14132789/relative-imports-for-the-billionth-time) – Brian61354270 Feb 06 '23 at 20:02
  • ^^ Even though the title says *Relative imports...* the very first thing the high-ranking answer covers is **script vs module** and that, I suspect, is where this issue really comes from. – kojiro Feb 06 '23 at 20:05
  • Aside: OP may be interested in Python [doctests](https://docs.python.org/3/library/doctest.html). Even though doctests are far more limited than the unittest module or pytest, they're a decent clean way to do testing in the same module as the code id written in, if that's your cup of tea. – kojiro Feb 06 '23 at 20:17
  • @brian, that post does not cover my question. With the "Try" block I am handling the case described on the answer for relative import when you call the script as "__main__". Something suspicious to me is that it works without problem on virtual env on my local machine, but I get the error when running from virtual env on server side – Javi M Feb 07 '23 at 08:36
  • 1
    You need to specify how you run your scripts (on local and on server). Which command do you use? From which *current working directory*? Is the project *installed* or run as-is? -- These are the rules I have written for myself, maybe they help you: https://sinoroc.gitlab.io/kb/python/python_imports.html – sinoroc Feb 07 '23 at 10:51
  • 1
    @JaviM Your main issue here is that you have an inconsistent package design. You're trying to expose the same modules as being a different locations, and you're trying to use the same source files as both scripts and modules. The best solution to your problem is to tidy your package structure, which would let you eliminate the try blocks altogether. The linked question should help you do this. It explains a few common confusions about scripts, modules, and directories, and how they relate to how Python searches for packages. – Brian61354270 Feb 07 '23 at 15:23
  • 1
    @JaviM Concretely, you can take these steps to settle on a consistent design: 1) separate your scripts (which are only ever run directly) from your modules (which need to be importable). 2) pick a directory to be the home for all of your importable code. The root directory of your project, or a top-level `src` directory are common choices. Let's call that directory `base`. 3) put `base` on the Python path. This is typically done either by installing your project as a distribution package, or by relying on `base` being your CWD [...] – Brian61354270 Feb 07 '23 at 15:29
  • 1
    [...] 4) move all of your importable modules to inside `base`, organized according to your desired package structure. So, for example, the file `base/dir1/dir2/foo.py` will become the module `dir1.dir2.foo`. 5) update _all_ of your imports to consistently use the package structure from (4). Always `import dir1.dir2.foo`, never `import foo`. 6) with `base` on your Python path, invoke your scripts as `python file.py`, and any runnable modules as `python -m dir1.dir2.file` – Brian61354270 Feb 07 '23 at 15:32
  • @Brian, Thaks for those tips. I followed the suggestion to put all "importable" files under "src". There, I have some scripts that are my bigger classes and and folder with some libs. On th root directory, if I understood right, the idea is to leave only "callable" scripts that won't be imported anywhere else. Thanks for the tip! – Javi M Feb 07 '23 at 15:57
  • @JaviM Sounds like a solid organization. Do be aware that if you're not using some sort of build system, using the `src` layout can have some complications, since you'll need to manually ensure that the `src` directory is on your Python path. This can be done by making it your CWD, but then you'll to invoke the scripts in your root directory as `python ../script.py`. This is solved automatically if you do use a [PEP 517 build system](https://peps.python.org/pep-0517/), since then you can just `pip install -e .` to add your `src` directory to your virtualenv's Python path. – Brian61354270 Feb 07 '23 at 16:05
  • 1
    For details on distribution packaging, there's [Packaging Python Projects](https://packaging.python.org/en/latest/tutorials/packaging-projects/) from the Python docs. There's a lot of build systems to choose from, both from the PSF and third parties. [Poetry](https://python-poetry.org/) is a particularly popular and full-featured one, though it also takes over dependency and virtual environment management. flit I think is the most minimal for just mapping a source tree to a package. Good luck! – Brian61354270 Feb 07 '23 at 16:07
  • @Brian, that will be interesting to fix. Currently the scripts under root directory are called from a crontab. My understanding is that it set the CWD as "/home". So if I set paths on the scripts using something like "os.getcwd()+", should do the trick and do not crash... – Javi M Feb 07 '23 at 16:09

0 Answers0