39

I have the following folder structure:

  • app
    • __init__.py
    • utils
      • __init__.py
      • transform.py
    • products
      • __init__.py
      • fish.py

In fish.py I'm importing transform as following: import utils.transform.

When I'm running fish.py from Pycharm, it works perfectly fine. However when I am running fish.py from the Terminal, I am getting error ModuleNotFoundError: No module named 'utils'.

Command I use in Terminal: from app folder python products/fish.py.

I've already looked into the solutions suggested here: Importing files from different folder, adding a path to the application folder into the sys.path helps. However I am wondering if there is any other way of making it work without adding two lines of code into the fish.py. It's because I have many scripts in the /products directory, and do not want to add 2 lines of code into each of them.

I looked into some open source projects, and I saw many examples of importing modules from a parallel folder without adding anything into sys.path, e.g. here: https://github.com/jakubroztocil/httpie/blob/master/httpie/plugins/builtin.py#L5

How to make it work for my project in the same way?

sergodeeva
  • 674
  • 1
  • 9
  • 15
  • Check if you are using a virtual environment in Pycharm by looking at the settings – Nicolò Gasparini Jun 07 '18 at 15:35
  • @NicolòGasparini I think am using virtualnev in Pycharm: in Project Interpreter I selected /application/venv/bin/python. I am using the same virtualnev in Terminal too... – sergodeeva Jun 07 '18 at 15:44
  • Can you show where your `__init__.py` files live? – Mad Physicist Jun 07 '18 at 15:45
  • Also show how you run your module from the command line. I'm guessing PyCharm is very smart about setting up your path. – Mad Physicist Jun 07 '18 at 15:47
  • Take a look at this https://stackoverflow.com/questions/1054271/how-to-import-a-python-class-that-is-in-a-directory-above, it should do what you need, however in the long run if your utils folder grows you may want to ensure it is in the path so you do not have to do relative imports. You may encounter this though: https://stackoverflow.com/questions/30669474/beyond-top-level-package-error-in-relative-import – d parolin Jun 07 '18 at 15:47
  • @MadPhysicist I have `__init__.py` inside utils and inside products – sergodeeva Jun 07 '18 at 15:47
  • You don't want to put one in apps? – Mad Physicist Jun 07 '18 at 15:48
  • @MadPhysicist in Terminal from the application folder I run: `python products/fish.py` – sergodeeva Jun 07 '18 at 15:48
  • @MadPhysicist yes, I also have __init__.py inside the application folder. – sergodeeva Jun 07 '18 at 15:49
  • You might also want to take a look ath his post: https://stackoverflow.com/questions/4383571/importing-files-from-different-folder?noredirect=1&lq=1 There is a thorough guide about Python imports by Chris Yeh: https://chrisyeh96.github.io/2017/08/08/definitive-guide-python-imports.html – meck93 Jun 07 '18 at 15:58
  • Please update your question with the new information on the folder structure, and an example of how you attempted to run your module. – Mad Physicist Jun 07 '18 at 16:19
  • Did you mean to have an `__init__.py` file in the `app` folder? Is `app` a package (so that the subpackages are `app.utils` and `app.products`? – Anakhand Nov 26 '20 at 11:29
  • 1
    Dupe of [Relative imports for the billionth time](https://stackoverflow.com/questions/14132789/relative-imports-for-the-billionth-time) – wim Aug 17 '21 at 16:55

2 Answers2

47

You probably want to run python -m products.fish. The difference between that and python products/fish.py is that the former is roughly equivalent to doing import products.fish in the shell (but with __name__ set to __main__), while the latter does not have awareness of its place in a package hierarchy.

Mad Physicist
  • 107,652
  • 25
  • 181
  • 264
21

This expands on @Mad Physicist's answer.

First, assuming app is itself a package (since you added __init__.py to it) and utils and products are its subpackages, you should change the import to import app.utils.transform, and run Python from the root directory (the parent of app). The rest of this answer assumes you've done this. (If it wasn't your intention making app the root package, tell me in a comment.)

The problem is that you're running app.products.fish as if it were a script, i.e. by giving the full path of the file to the python command:

python app/products/fish.py

This makes Python think this fish.py file is a standalone script that isn't part of any package. As defined in the docs (see here, under <script>), this means that Python will search for modules in the same directory as the script, i.e. app/products/:

If the script name refers directly to a Python file, the directory containing that file is added to the start of sys.path, and the file is executed as the __main__ module.

But of course, the app folder is not in app/products/, so it will throw an error if you try to import app or any subpackage (e.g. app.utils).

The correct way to start a script that is part of a package is to use the -m (module) switch (reference), which takes a module path as an argument and executes that module as a script (but keeping the current working directory as a module search path):

If this option is given, [...] the current directory will be added to the start of sys.path.

So you should use the following to start your program:

python -m app.products.fish

Now when app.products.fish tries to import the app.utils.transform module, it will search for app in your current working directory (which contains the app/... tree) and succeed.

As a personal recommendation: don't put runnable scripts inside packages. Use packages only to store all the logic and functionality (functions, classes, constants, etc.) and write a separate script to run your application as you wish, putting it outside the package. This will save you from this kind of problems (including the double import trap), and has also the advantage that you can write several run configurations for the same package by just making a separate startup script for each.

Anakhand
  • 2,838
  • 1
  • 22
  • 50