Issue with Relative Import

Question

I am having error in importing both via relative and absolute import. Something I have done many times in the past and it used to work.

here's my folder structure:

project
├── src
│   └── notebook.ipynb
    |_____ test.py
    └──  __init__.py

So basically within src folder (which is made a package using init.py) I have two files- notebook.ipynb and test.py

Now within notebook.ipynb, in cell I do:

from . import test 

Gives the following error:

ImportError                               Traceback (most recent call last)
Cell In[13], line 1
----> 1 from . import test

ImportError: attempted relative import with no known parent package

Even if I try absolute import:

from src import test

I get:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[14], line 1
----> 1 from src import test

ModuleNotFoundError: No module named 'src'

Now I have used similar importing styles in past projects and it would work all fine. Within the same package I can either via relative or absolute. It doesn't NEED me to specify any PYTHONPATH in my env variable.

But this time it isn;t working.

I am using Python 3.9.15 on MAC OS installed within a conda env

Answer 1

Short answer: You should not put your notebook in your src folder. The proper way of working with package is to install them then use it as an external package (like you do with numpy, torch, pandas, matplotlib etc..).

Long answer: The src layout is meant to contain only importable modules in Python. There are several reasons, I let you read the exhaustive and short list of reasons in the link. So basically your project tree should like this:

project
├── notebooks
│   └── notebook.ipynb

├── src
    ├── package_name
        ├── test.py
        └── __init__.py

I let you read the short tutorial to package your project.

Although, when you explore data, testing some processing and so on, this packaging step can be repetitive. It means that each time you make changes in your package you have to install it again and again. To solve this problem you can install it as a editable package.

Note: if you use a notebook to test / develop / use your package, you need to restart the kernel each time you make changes in your package though, editable mode set or not. If your package is not installed as editabled then you should restart the kernel and install the package for every change in your package's code.

Note 2: all links provided are from the official authority of packaging in Python. It does not mean this that it's the only way to go through it, but, at least, this is some common practices for clean Python project.

Edit after the comment: Also, you can use your text editor / IDE to change how Python will search for module. It will heavily depends of your editor / IDE. In my case, if I open the project VSCode and I set the working directory option as the opened folder, you can import what you want. Still, do not place your notebook in your src folder. Notebooks are not source file, even if they are going to by converted to script. If you use VSCode, I can add more details how to set it up as a workaround. But this way of working is not standard so I do not recommand it.