How to install a custom libarry in Jupyter notebook of GCP?

Question

I am developing a Python library and I need to make it available from GCP Notebook.

Is it possible? How?

Details:

I use Pipenv to manage my library dependencies. Currently my library source code exists in local and in a private git repository. So it is not in PyPI.
My code has multiple module files in nested directories.
My library's dependencies exist in PyPI.
Using Pipenv, the dependencies are described in Pipefile.
This is the type of my Jupyter VM instance : https://cloud.google.com/deep-learning-vm
And this is some interesting structure I could find using SSH from Google console :

$ ls /opt/deeplearning/ 
bin  binaries  deps  jupyter  metadata  proxy-agent-config.json  restriction  src  workspace

I envisage to install my library (using pip or something else) to be able to import its modules from the notebooks.
I need that all the dependencies of my library to be installed when installing the library.
If the Python Packages Index is public, I don't want to publish my library in it being proprietary.

Thank you.

Where is your custom library? On PyPI? In a git repo somewhere? — Dustin Ingram, Mar 23 '20 at 15:45
In my local machine and in a proprietary (not public) git repository — Farah, Mar 23 '20 at 15:50
Hi, could you provide more detail about your library? There multiple alternatives to installing packages on GCP, such as Terminal or in a Code cell. are you considering to enable your library to install trough pip? — Yor Jaggy, Mar 23 '20 at 16:21
How can you authenticate with your git repo? Can you use `git+ssh://` or `git+https://`? — Dustin Ingram, Mar 23 '20 at 19:10

score 1 · Answer 1 · answered Mar 24 '20 at 08:29

What I understood from your question is: you are writing your own python module, which depends on many third-part python packages (can be installed with pip).

In this situation, I would probably do a pip freeze on the actual environment where the module loads everything perfectly.

pip freeze > requirements.txt (It will create a requirements.txt file with all the dependency modules/libraries)

Now, once in the jupyter notebook, you can use the following command to first install all the requirements.

(Run the following in the notebook code cell)

# Install a pip package in the current Jupyter kernel
import sys
!{sys.executable} -m pip install -r requirements.txt

How to install a custom libarry in Jupyter notebook of GCP?

1 Answers1