5

My directory structure is as follows.

project/
  jupyter/
    note.ipynb
  src/
    some_script.py
    another_script.py

The jupyter subdirectory contains Jupyter notebooks and the src subdirectory contains my .py files. The src subdirectory is marked as the sources root.

In the Jupyter notebook note.ipynb, if I do import some_script, then I get ImportError: No module named 'some_script'. However, from src import some_script works fine, but not if some_script also has a line import another_script. In my mind I'm thinking that there is a setting or switch somewhere that adds the src subdirectory to the PYTHONPATH used by the Jupyter Notebook environment by default, but I don't know how to do it, could you help?

I know that I can always use sys.path.append but I don't want to do that in every single Jupyter notebook I create. I know also that I can just put the .ipynb files and .py files in the same subdirectory but I don't like organising my work that way. Finally, I know I can just set the project directory as my sources root instead of the src subdirectory, but that would mean that I'd have to do from src import ... everywhere in my .py scripts, and obviously I don't want to do that. Is there a solution to this problem as I've described above?

Ray
  • 7,833
  • 13
  • 57
  • 91

2 Answers2

2

In my mind I'm thinking that there is a setting or switch somewhere that adds the src subdirectory to the PYTHONPATH used by the Jupyter Notebook environment by default

Yes, you can set ENV variables per kernel, but you'll need to find the place where those are stored.

You can run jupyter kernelspec list for that, then open up a kernel.json in those.

For example, this one is from my PySpark Jupyter kernel.

See the env: { "PYTHONPATH": variable. 

{
  "argv": [
    "/Users/name/Library/Jupyter/kernels/apache_toree_pyspark_16/bin/run.sh",
    "--profile",
    "{connection_file}"
  ],
  "env": {
    "DEFAULT_INTERPRETER": "PySpark",
    "__TOREE_SPARK_OPTS__": "",
    "__TOREE_OPTS__": "",
    "SPARK_HOME": "/usr/local/opt/apache-spark@1.6/libexec",
    "PYTHONPATH": "/usr/local/opt/apache-spark@1.6/libexec/python:/usr/local/opt/apache-spark16/libexec/python/lib/py4j-0.9-src.zip",
    "PYTHON_EXEC": "python"
  },
  "display_name": "Apache Toree - PySpark (1.6)",
  "language": "python"
}
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • I see. So there is no "elegant" way of doing this in PyCharm, for example by checking a box or something? – Ray Mar 13 '17 at 16:31
  • I mean, you could set `PYTHONPATH` within PyCharm GUI, yes. http://stackoverflow.com/questions/34685905/how-to-link-pycharm-with-pyspark – OneCricketeer Mar 13 '17 at 17:31
0

Dunno if this helps but Jupyter restricts itself to looking only at files below the place it was opened in, this is as far as I know , by design to help make serving Jupyter Notebooks more secure. In multi user environments by default.

dartdog
  • 10,432
  • 21
  • 72
  • 121