2

I'm trying to configure jupyterlab dask extension so that the "New" cluster button will create a KubeCluster instead of the default LocalCluster.

Tried to edit ~/.config/dask/labextension.yml so it will have this content: 

kubernetes:
  worker-template:
    metadata:
    spec:
      restartPolicy: Never
      containers:
      - args:
          - dask-worker
          - --nthreads
          - '2'
          - --no-bokeh
          - --memory-limit
          - 12GB
          - --death-timeout
          - '60'
        image: ${JUPYTER_IMAGE_SPEC}
        name: dask-${JUPYTERHUB_USER}
        resources:
          limits:
            cpu: "1.75"
            memory: 12G
          requests:
            cpu: 1
            memory: 12G

labextension:
  factory:
    module: dask_kubernetes
    class: KubeCluster
    args: []
    kwargs: {}

(source: https://github.com/pangeo-data/pangeo-cloud-federation/blob/8f7f4bf9963ef1ed180dd20c952ff1aa8df54ca2/deployments/ocean/image/binder/dask_config.yaml#L37-L42)

Yet creating a new cluster using the notebook button still creates a LocalCluster. What am I missing?

user3599803
  • 6,435
  • 17
  • 69
  • 130

1 Answers1

0

What you've done above seems fine to me.

Have you restarted JupyterLab? It reads the configuration files at startup time.

MRocklin
  • 55,641
  • 23
  • 163
  • 235
  • How to restart? I used `dask.config.refresh()`. And I tried creating a new notebook. Both didnt help – user3599803 Feb 23 '20 at 18:26
  • You need to restart JupyterLab, not Dask. – MRocklin Feb 23 '20 at 18:31
  • Is it possible from a notebook code? Because I can't restart the kubernetes pod – user3599803 Feb 23 '20 at 19:05
  • I don't think so. If you are using a service like JupyterHub then you want to stop your server, and then restart it. There should be a button for this somewhere. For more information, I encourage you to ask questions of whoever set up your Jupyter-in-Kubernetes system. – MRocklin Feb 23 '20 at 20:05