0

This must have been asked before, but I cannot for the life of me find it.

I have a project structure something like this 

ROOT
├── README.md
├── modules
│   ├── __init__.py
│   ├── mod1.py
│   └── mod2.py
├── tests
│   ├── test_mod1.py
│   └── test_mod2.py
├── notebooks
│   ├── nb1.ipynb
│   ├── nb2.ipynb
│   └── sub_dir
│       ├── sub_nb.ipynb
│       ├── generate_py
│       └── py_files
│           └── sub_nb.py
├── definitions
└── main.py

So from main.py I am able to import anything from definitions, or any module from ROOT/modules.

What I want is to be able to import from these from anywhere within the notebooks directory tree. I know I could do this using:

import sys
sys.path.append("..")

But the notebooks directory tree has many layers, and I don't want my code to start looking like this:

import sys
sys.path.append("../../../../")

What's more, the file generate_py is a bash script, that converts the jupyter notebooks (.ipynb) to .py files and stashes the .py files into the ./py_files subdir.

With the above method i end up having to manually edit every file to put an extra ../ into the sys.append(). This is annoying.

If I run files from within pycharm all works well (I'm guessing it updates your PYTHONPATH to include the project root when you create main.py?)

To further complicate, this project is run on several machines, shared via git. So absolute references are out.

But running from terminal, or from within jupyter it cant gind modules, or definitions without going through the sys.append() process. And I feel there must be a better way.

So what is the best practice here?

FinleyGibson
  • 911
  • 5
  • 18
  • Instead of adding ROOT to your `PYTHONPATH` dynamically in every file, every time it runs, you can add it by modifying the environment variable, i.e. add `export PYTHONPATH="${PYTHONPATH}:/path/to/ROOT"` to your `.bashrc` or similar. Since ROOT might be in different locations on different users' machines, you should either have a README telling users to do this, or else a `setup` shell script which [computes the path](https://stackoverflow.com/a/246128/6445069) and does this automatically. – Phydeaux Mar 08 '19 at 11:48
  • I did mean `generate_py`. – FinleyGibson Mar 08 '19 at 17:01
  • I did think about editing PYTHONPATH in .bashrc, but it still seems a bit inelegant. I will then forever expose any project I work on to accidentally importing from `definitions`, when I inevitably forget to remove the line later. I still feel there must be a cleaner way. – FinleyGibson Mar 08 '19 at 17:04
  • You can use `virtualenv` (optionally also `virtualenvwrapper` for a nicer experience) to make an environment on a per-project basis, and add the `export PYTHONPATH...` to the environment's `activate` script instead, to avoid polluting your `.bashrc` – Phydeaux Mar 08 '19 at 17:14

0 Answers0