Setting a default sys.path for a Notebook

Question

I have all my .py files inside a folder script and all my IPython-notebooks under a folder named Notebook.

There are multiple cross dependencies for each notebook file on one or more files on script.

Having sys.path.append on top of every notebook seems cumbersome and I am hoping there is a way to add a default lookup path just like we add PYTHONPATH to .bash_profile.

Now I do the following:

import sys
sys.path.append("<path where DeriveFinalResultSet.py exists>)
import DeriveFinalResultSet as drs

I wish to have a setting where I can do the below:

import DeriveFinalResultSet as drs

I don't know the answer here, but perhaps you could add it to a customization file: https://ipython.org/ipython-doc/2/config/intro.html#setting-configurable-options — mgilson, Jul 07 '16 at 04:11
Alternatively, you could create a simple bash script with sets PYTHONPATH before invoking `ipython` on the given script... — mgilson, Jul 07 '16 at 04:12
@mgilson: I updated .bash_profile and pointed PYTHONPATH to point to the directory I wanted. And I fire up `jupiter-notebook` from console and it seems to be doing the trick. I am just posting the answer below. If you could, can you verify if it is right? — Sreejith Menon, Jul 10 '16 at 03:20

score 11 · Accepted Answer · edited May 28 '17 at 12:57

To avoid "hidden configurations" (i.e. things that aren't in source control/machine-specific) and to maintain a notebook/code separation like you describe, I do something like the below:

code/
    mymodule.py
    mypackage/
        __init__.py

notebooks/
    mynb.ipynb
    mynb2.ipynb
    paths.py   <--- below

In paths.py:

import sys
import pathlib
sys.path.insert(0, str(pathlib.Path(__file__).parents[1] / 'code'))
# sys.path[0] = str(pathlib.Path(__file__).parents[1] / 'code')

Then in mynb*.ipynb I can happily do:

import paths
import mymodule, mypackage

, etc.

The latter form effectively replaces the import path from the empty-string (current directory) to the "code" directory, which is perhaps a bit cleaner. This makes imports insensitive to using stuff like os.chdir().

Great answer. Maybe `pathlib.Path().resolve().parent` as a minor refinement? — aparkerlue, Sep 05 '17 at 21:52

score 2 · Answer 2 · answered Jun 22 '17 at 07:59

I wrote simple bash script which updates the path and launches Jupyter:

#!/usr/bin/env bash

echo "Saving PYTHONPATH"
ORIGINAL_PYTHONPATH=$PYTHONPATH
echo "Prepending package to PYTHONPATH"
export PYTHONPATH="$PWD/:$ORIGINAL_PYTHONPATH"
echo "Starting Jupyter"
jupyter notebook
echo "Reverting to the original PYTHONPATH"
export PYTHONPATH=$ORIGINAL_PYTHONPATH

score 0 · Answer 3 · answered Jul 10 '16 at 03:25

After some research I realized changing PYTHONPATH in .bash_profile should do the trick.

Here are the two lines that I added to my .bash_profile

PYTHONPATH="<path where DeriveFinalResultSet.py exists>:$PYTHONPATH"
export PYTHONPATH

To verify, I did the following after opening a fresh IPython Notebook.

import sys
print(sys.path)
['', '**<path where DeriveFinalResultSet.py exists>**', '<some path>Google Drive/Project/AnimalPhotoBias/Notebooks', '<some path>anaconda/lib/python35.zip', '<some path>anaconda/lib/python3.5', '<some path>anaconda/lib/python3.5/plat-darwin', '<some path>anaconda/lib/python3.5/lib-dynload', '<some path>anaconda/lib/python3.5/site-packages/Sphinx-1.3.5-py3.5.egg', '<some path>anaconda/lib/python3.5/site-packages/setuptools-20.3-py3.5.egg', '<some path>anaconda/lib/python3.5/site-packages', '<some path>anaconda/lib/python3.5/site-packages/aeosa', '<some path>anaconda/lib/python3.5/site-packages/IPython/extensions', '<some path>.ipython']

This should work. You could simplify it to 1 line: `export PYTHONPATH="...:${PYTHONPATH}"` — mgilson, Jul 10 '16 at 04:17
Probably, `bash_profile` — is not the best place for such config. If you have two or more projects this approach will add too many extra paths to `PYTHONPATH`, and could lead to name-conflict. — maxkoryukov, May 28 '17 at 09:11
You are right, @maxkoryukov. This will not be a good approach when handling multiple projects. Thank you! — Sreejith Menon, May 28 '17 at 17:21

Setting a default sys.path for a Notebook

3 Answers3

Linked