Stack Overflow
Stack Overflow

Questions tagged [dask-gateway]

12 questions
5
votes
1 answer

Dask Gateway, set worker resources

I am trying to set the resources for workers as per the docs here, but on a set up that uses Dask Gateway. Specifically, I'd like to be able to follow the answer to this question, but using Dask Gateway. I haven't been able to find a reference to…
bill_e
  • 930
  • 2
  • 12
  • 24
3
votes
0 answers

How to View Dask Daskboard in Dask Gateway when using a private IP address/VPC?

We deployed Dask Gateway on Kubernetes on Google Cloud Platform. We are currently using an internal TCP load balancer to expose the traefik proxy for security purposes. Our users are able to create a client connection to the cluster generated…
Riley Hun
  • 2,541
  • 5
  • 31
  • 77
2
votes
1 answer

How do you integrate GPU support with Dask Gateway?

We are currently using Dask Gateway with CPU-only workers. However, down the road when deep learning becomes more widely adopted, we want to transition into adding GPU support for the clusters created through Dask Gateway. I've checked the Dask…
Riley Hun
  • 2,541
  • 5
  • 31
  • 77
1
vote
0 answers

Connection to dask gateway server goes down after a while with JupyterHub authentication

I am doing a multi user setup with DaskHub and JupyterHub with the use of Daskgateway servers but my connection to dask-gateway servers fails with error after a while: Error [I 2023-04-24 09:51:14.477 DaskGateway] Creating cluster…
Jyoti Singh
  • 33
  • 5
1
vote
1 answer

Dask-gateway - sending self-defined python files to worker

I am using Dask in a cluster environment through Dask-gateway. I created the computation graph in a Jupyter Notebook. Within my delayed function, it calls multiple functions defined in several separate .py files. Currently, I am running into the…
Undecided
  • 611
  • 8
  • 13
1
vote
0 answers

Dask Gateway Setup on Azure

I am trying to setup dask gateway in AKS. Following the documentation I was able to start the dask gateway server in AKS. We have also hosted a separate jupyternotebook instace within the same cluster. When I try to access the gateway server from…
aquero
  • 843
  • 4
  • 13
  • 30
1
vote
0 answers

Dask - new cluster creation fails, HDFS files owed by "dask" user

I have setup dask on my MapR cluster's edge node following the directions here: https://gateway.dask.org/install-hadoop.html Per those directions, I'm testing the install by running the following in a JupyterHub spawned ipython notebook: from…
Matt Mc
  • 11
  • 1
0
votes
0 answers

Dask Gateway AttributeError: can't set attribute

I'm a newbie to dask gateway, trying to learn how to use it. I am just trying to open a dataframe and compute it. Something like: auth = BasicAuth(username="my_user", password="my_password") gateway = Gateway("http://my_path", auth=auth) options =…
Rafael Higa
  • 655
  • 1
  • 8
  • 17
0
votes
1 answer

can not connect to AKS daskhub

I followed this readme to deploy dusk on my AKS test cluster: helm values changes: set the api token changed traefik to load banacher gateway.prefix: "/services/dask-gateway" ended up with this services in aks: api-test-dask-gateway …
Jester
  • 3,069
  • 5
  • 30
  • 44
0
votes
0 answers

Configuring dask gateway via env vars

In dask gateway's config docs, there is a setting adaptive period. How does this interact with the standard dask.config variable DASK_DISTRIBUTED__ADAPTIVE__INTERVAL? In general, the GatewaySchedulerService seems to have a parameters that mimic…
HoosierDaddy
  • 720
  • 6
  • 19
0
votes
1 answer

Dask Gateway - Dask Workers Dying Due to PermissionError

I am trying to deploy Dask Gateway on Google Kubernetes Engine. No issues w/ the deployment. However, I am experiencing issues when using a custom dask-gateway dockerfile that inherits from the default docker image from dockerhub; the container is…
Riley Hun
  • 2,541
  • 5
  • 31
  • 77
0
votes
1 answer

Dask Hub/JupyterHub - Cannot start Python Kernel

I deployed Dask Hub (Dask Gateway + Jupyterhub) on Google Kubernetes Engine using helm. I am experiencing trouble fetching the Python Kernel when I start up my jupyter notebook instance. This error occurs when I am on my company's VPN, but no error…
Riley Hun
  • 2,541
  • 5
  • 31
  • 77