Specify zmq file ports when starting jupyter or connect to existing kernel using connection file

Question

I would like to use the Docker image jupyter/datascience-notebook to start a Jupyter notebook frontend, but I need to be able to control which ports it chooses to use for its communication. I understand that the server is designed to potentially provision many kernels, not just one, but what I want is for the first kernel to use the ports I specify. I have tried supply arguments like:

docker run --rm -it jupyter/datascience-notebook:latest start-notebook.sh  --existing /my/connection/file.json
docker run --rm -it jupyter/datascience-notebook:latest start-notebook.sh  --KernelManager.control_port=60018

And it does not seem to care, instead creating the connection file in the usual location under /home/jovyan/.local/share/jupyter/

Any assistance is appreciated.

Answer 1

I ended up doing as suggested in a similar question - IPython notebook: How to connect to existing kernel? this, I could not find a better way.

Subclass LocalProvisioner to override its pre_launch method

from typing import Any, Dict

from jupyter_client import LocalProvisioner, LocalPortCache, KernelProvisionerBase
from jupyter_client.localinterfaces import is_local_ip, local_ips


class PickPortsProvisioner(LocalProvisioner):

  async def pre_launch(self, **kwargs: Any) -> Dict[str, Any]:
    """Perform any steps in preparation for kernel process launch.
    This includes applying additional substitutions to the kernel launch command and env.
    It also includes preparation of launch parameters.
    Returns the updated kwargs.
    """

    # This should be considered temporary until a better division of labor can be defined.
    km = self.parent
    if km:
      if km.transport == 'tcp' and not is_local_ip(km.ip):
        raise RuntimeError(
          "Can only launch a kernel on a local interface. "
          "This one is not: %s."
          "Make sure that the '*_address' attributes are "
          "configured properly. "
          "Currently valid addresses are: %s" % (km.ip, local_ips())
        )
      # build the Popen cmd
      extra_arguments = kwargs.pop('extra_arguments', [])

      # write connection file / get default ports
      # TODO - change when handshake pattern is adopted
      if km.cache_ports and not self.ports_cached:
        lpc = LocalPortCache.instance()
        km.shell_port = 60000
        km.iopub_port = 60001
        km.stdin_port = 60002
        km.hb_port = 60003
        km.control_port = 60004
        self.ports_cached = True

      km.write_connection_file()
      self.connection_info = km.get_connection_info()

      kernel_cmd = km.format_kernel_cmd(
        extra_arguments=extra_arguments
      )  # This needs to remain here for b/c
    else:
      extra_arguments = kwargs.pop('extra_arguments', [])
      kernel_cmd = self.kernel_spec.argv + extra_arguments

    return await KernelProvisionerBase.pre_launch(self, cmd=kernel_cmd, **kwargs)

Specify entry point in setup.py

entry_points = {
    'jupyter_client.kernel_provisioners': [
        'pickports-provisioner = mycompany.pickports_provisioner:PickPortsProvisioner',
    ],
},

Create kernel.json to overwrite the default one

{
 "argv": [
  "/opt/conda/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3 (ipykernel)",
 "language": "python",
 "metadata": {
  "debugger": true,
  "kernel_provisioner": { "provisioner_name": "pickports-provisioner" }
 }
}

Dockerfile

# Start from a core stack version
FROM jupyter/datascience-notebook:latest

# Install from requirements.txt file
COPY --chown=${NB_UID}:${NB_GID} requirements.txt .
COPY --chown=${NB_UID}:${NB_GID} setup.py .
RUN pip install --quiet --no-cache-dir --requirement requirements.txt

# Copy kernel.json to default location
COPY kernel.json /opt/conda/share/jupyter/kernels/python3/

# Install from sources
COPY --chown=${NB_UID}:${NB_GID} src .
RUN pip install --quiet --no-cache-dir  .

Profit???