I'd like to include code from another file in another Jupyter .ipynb file on the AWS Elastic MapReduce platform.
But many of the methods I have seen online for including utility functions / common Python code that would seem to work outside of AWS, don't work inside the hosting environment for EMR-based Notebooks. I'm assuming this is file system security/server restrictions. If someone knows of an example of including code from a .py file and/or .ipynb file from the same directory that works on AWS I would love to see an example.
This method did not work for me. The find_notebook returns None. https://jupyter-notebook.readthedocs.io/en/4.x/examples/Notebook/rstversions/Importing%20Notebooks.html
or this library / method ipynb import another ipynb file
Is there an AWS "approved" or recommended way of including common Python/PySpark code into a Jupyter Notebook?
Note: this question How to import from another ipynb file in EMR jupyter notebook which runs a PySpark kernel? I have ssh'd into the master server and installed different packages like ipynb. Didn't work even though the module installed fine. And the environment can see it. But the overall technique did not work.
Error:
An error was encountered:
"'name' not in globals"
Traceback (most recent call last):
KeyError: "'name' not in globals"
