1

I am trying to create a package with multiple .py files. When I try to import a file from the package from a script in the parent directory, a second import between files within the package fails with ModuleNotFoundError. What is going on here, what am I doing wrong, and how do I fix this?

My file layout:

python_path_test/
    parent.py
    mypackage/
        __init__.py
        child.py
        sibling.py

parent.py:

import mypackage.child
mypackage.child.foo()

subdir/__init__.py is empty

mypackage/child.py:

import sibling
def foo():
    print("foo")
    sibling.bar()

mypackage/sibling.py:

def bar():
    print("bar")

Now, when I try to run parent.py, I get a ModuleNotFoundError:

C:\>cd \python_path_test
C:\python_path_test>py -3 parent.py
Traceback (most recent call last):
  File "C:\python_path_test\parent.py", line 1, in <module>
    import mypackage.main
  File "C:\python_path_test\mypackage\main.py", line 1, in <module>
    import sibling
ModuleNotFoundError: No module named 'sibling'

I expected import sibling in child.py to succeed because sibling.py is in the same directory as child.py.

In C and C++ the directory containing the current source file is always the first entry in the search path, but clearly that is not what is going on here, since import sibling in child.py is failing even though both files are in the same directory. I must be doing something silly, because cross-importing files that are in the same directory is the most basic thing you'd want to do when splitting code into multiple files. What is the correct way to proceed?

EDIT (revised)

Mike suggested adding a dot, to use relative imports (which I am not familiar with). But when I changed child.py to use import .sibling I got a syntax error. Karl and Varshitha both pointed out that I should write from . import sibling -- that works for the example given above, but then it breaks running scripts from the command line in the package directory. For example

mypackage/script.py

from . import child
child.foo()

when run, outputs:

C:\python_path_test>cd mypackage
C:\python_path_test>py -3 script.py
Traceback (most recent call last):
  File "C:\python_path_test\mypackage\script.py", line 1, in <module>
    from . import child
ImportError: attempted relative import with no known parent package

This same breakage occurs if I want to run files in mypackage that contain if __name__ == '__main__': sections (when they use relative imports). I have seen explanations of why this error occurs (files run as scripts do not set a parent package name) but I do not know the correct way to resolve this. It seems that if I use relative imports, I lose the ability to run scripts inside my package directory. I have discovered that I can use the -m flag, and run the script from the parent directory using:

cd python_path_test
py -3 -m mypackage.script

but that is rather unintuitive and inconvenient, which makes me think that I am still doing something wrong.

Furthermore, if I have a Jupyter notebook in mypackage/ containing the single cell import child or from . import child it fails with the same error and I know of no workaround.

Ross Bencina
  • 3,822
  • 1
  • 19
  • 33
  • 1
    Looks like you forgot a dot: [what does a . in an import statement in Python mean?](https://stackoverflow.com/questions/7279810/what-does-a-in-an-import-statement-in-python-mean) – Mike 'Pomax' Kamermans Apr 02 '23 at 04:34
  • A dot? I don't see any missing dots so I must be missing something. Are you suggesting that `child.py` should contain `import .sibling`? – Ross Bencina Apr 02 '23 at 04:37
  • It is helpful and enlightening but I have said "No" to https://stackoverflow.com/questions/7279810/what-does-a-in-an-import-statement-in-python-mean being an answer to my question, because it is not clear to me whether relative imports are the appropriate solution to my problem (never heard of them before and they seem quite new, having only become standard in 2.7) or whether relative imports are the most pythonic way to handle my issue. I would be happy for someone to answer "use relative imports" with an explanation then I can see whether that answer is upvoted by people with expertise. – Ross Bencina Apr 02 '23 at 04:51
  • 1
    Relative imports can **only** use the `from` import syntax. For the current directory, that looks like `from . import sibling`. – Karl Knechtel Apr 02 '23 at 05:02
  • 2
    I'm tired of not having a **ready, proper** canonical for this **very common** question (see related meta discussion: https://meta.stackoverflow.com/questions/423857) and this question seems about as good as one of the ones I figured I'd have to create myself; so I'm going to go ahead and give my best answer here. – Karl Knechtel Apr 02 '23 at 05:05
  • @KarlKnechtel would love to see your answer. No accepted answer right now. – Ross Bencina Apr 02 '23 at 05:10
  • I'm having second thoughts in fact... what I'm trying to write up at the moment goes beyond the *apparent* scope of this question (and will probably be pretty long). But yes, I do advise using relative imports for your situation. – Karl Knechtel Apr 02 '23 at 05:40
  • @KarlKnechtel I just updated my "EDIT" to the question to point out that when I try to use relative imports it breaks scripts that I have in the package (e.g. scripts I use for ad-hoc testing and development) – Ross Bencina Apr 02 '23 at 05:41
  • Yes, there **is** a canonical for *that part*. Multiple contenders, in fact. I've now given you the one that I think is phrased most similarly to your particular situation, since that's where we've ended up. I do think I still need to write my own question; I'll be happy to let you know about it when it's ready - this is going to take quite a bit of work. – Karl Knechtel Apr 02 '23 at 06:08
  • @KarlKnechtel So there is no solution to my problem? Guido says don't run scripts from a package directory? Note that I have made a further update which points out that the `-m` workaround does not work for Jupyter notebooks in the package directory. – Ross Bencina Apr 02 '23 at 06:15
  • "Note that I have made a further update which points out that the -m workaround does not work for Jupyter notebooks in the package directory" - no, you haven't; you've said that the *absolute* import didn't work. I don't have any experience with Jupyter notebooks, and dealing with them is a separate question. Anyway, the `-m` "workaround" *is* the solution to your problem, if you insist on leaving the script inside the package. Also, your `parent.py` - which appeared to be the intended "driver" script - **isn't** inside the package, so I couldn't have originally anticipated that question. – Karl Knechtel Apr 02 '23 at 06:21
  • @KarlKnechtel it is the same error with Jupyter whether I use absolute or relative imports. Running scripts and running notebooks are two standard usecases that I use in combination with developing "library" code in .py files. All I want to do is organise things into directories (which means packages in python) but as soon as I use more than one directory I hit the problems outlined in this question. – Ross Bencina Apr 02 '23 at 06:25
  • @KarlKnechtel I can see that you didn't anticipate it, but there may be many drivers, `parent.py` is one of them. I didn't anticipate that the answer to the pre-edit part of my question would break the rest of my codebase. – Ross Bencina Apr 02 '23 at 06:28

2 Answers2

0

In your child.py, put the following code:

try:
    import sibling
    
except:
    from mypackage import sibling

def foo():
    print("foo")
    sibling.bar()
Ross Bencina
  • 3,822
  • 1
  • 19
  • 33
Dinux
  • 644
  • 1
  • 6
  • 19
-1

You may have an issue with your Python package's import structure.

Make sure your package has an __init__.py file in the top-level directory.

When importing modules from within the package, use a relative import instead of an absolute import. For example, instead of using import mypackage.mymodule, use from . import mymodule. This ensures that Python looks for the module within the package's directory structure, rather than trying to import it globally. Relative imports are described in PEP-328.

If you're running your script from outside the package directory, make sure you're adding the package directory to your Python path. You can do this using the sys.path.append() method in your script. For example, if your package is located at /path/to/mypackage, you would add sys.path.append('/path/to') to your script.

Ross Bencina
  • 3,822
  • 1
  • 19
  • 33
  • `mypackage` does have an `__init__.py` the file `parent.py` in `python_path_test` is just for testing and development, it's not part of the package. – Ross Bencina Apr 02 '23 at 04:45
  • 1
    **This is wrong**. Since 3.3, `__init__.py` is **not needed** simply to support relative import. There are multiple reasons why someone might create a blank `__init__.py` (and it also can contain useful code), but this is not one of them. – Karl Knechtel Apr 02 '23 at 05:15