Stack Overflow
Stack Overflow

Questions tagged [databricks-cli]

67 questions
11
votes
3 answers

Create a new cluster in Databricks using databricks-cli

I'm trying to create a new cluster in Databricks on Azure using databricks-cli. I'm using the following command: databricks clusters create --json '{ "cluster_name": "template2", "spark_version": "4.1.x-scala2.11" }' And getting back this…
Mor Shemesh
  • 2,689
  • 1
  • 24
  • 36
8
votes
2 answers

Create Azure Key Vault backed secret scope in Databricks with AAD Token

My ultimate goal is to mount ADLS gen2 containers into my Databricks workspace as part of my Terraform-managed deployment under the auspices of an Azure Service Principal. This is a single deployment that creates all the Azure resources (networking,…
Xophmeister
  • 8,884
  • 4
  • 44
  • 87
8
votes
3 answers

Databricks CLI: SSLError, can't find local issuer certificate

I have installed and configured the Databricks CLI, but when I try using it I get an error indicating that it can't find a local issuer certificate: $ dbfs ls dbfs:/databricks/cluster_init/ Error: SSLError:…
James Adams
  • 8,448
  • 21
  • 89
  • 148
7
votes
4 answers

How do I use databricks-cli without manual configuration

I want to use databricks cli: databricks clusters list but this requires a manual step that requires interactive work with the user: databricks configure --token Is there a way to use databricks cli without manual intervention so that you can run…
Mor Shemesh
  • 2,689
  • 1
  • 24
  • 36
6
votes
2 answers

How do we access databricks job parameters inside the attached notebook?

In Databricks if I have a job request json as: { "job_id": 1, "notebook_params": { "name": "john doe", "age": "35" } } How do I access the notebook_params inside the job attached notebook?
Sannix19
  • 75
  • 1
  • 6
6
votes
4 answers

'databricks configure --token' hangs for input

I run following task in Azure DevOps, it always hangs for input? Why wasn't my bash automatic supply working? databricksUrl=https://... databricksToken=***** databricks configure --token << EOF $(databricksUrl) $(databricksToken) EOF
user3023949
  • 121
  • 2
  • 8
6
votes
2 answers

Databricks CLI Configuration Error "Please reconfigure with dbfs configure"

I've installed and configured the Databricks CLI but get the error below when running databricks workspace ls Error Returned: Error: Your authentication information may be incorrect. Please reconfigure with dbfs configure Steps Used to Install…
ChaseHardin
  • 2,189
  • 7
  • 29
  • 50
4
votes
5 answers

How to configure Databricks token inside Docker File

I have a docker file where I want to Download the Databricks CLI Configure the CLI by adding a host and token And then running a python file that hits the Databricks token I am able to install the CLI in the docker image, and I have a working…
DataTx
  • 1,839
  • 3
  • 26
  • 49
4
votes
1 answer

How to create a databricks job with parameters

I'm creating a new job in databricks using the databricks-cli: databricks jobs create --json-file ./deploy/databricks/config/job.config.json With the following json: { "name": "Job Name", "new_cluster": { "spark_version":…
Mor Shemesh
  • 2,689
  • 1
  • 24
  • 36
3
votes
1 answer

How do I find out which Databricks groups I'm a member of?

I'm trying to figure out if I'm a member of a specific group on Databricks using either the website or Databricks CLI. I found a reference to managing groups via an account console, but this requires an admin account. I also found the Databricks…
savagedata
  • 713
  • 1
  • 5
  • 10
3
votes
1 answer

"No Isolation Shared" Databricks job cluster through CLI

I turned on Unity Catalog for our workspace. Now a job cluster has an access mode setting. (docs) I can manually change this setting on the UI: But how do I control this setting when creating the job through databricks jobs create --json-file…
Daniel Darabos
  • 26,991
  • 10
  • 102
  • 114
3
votes
2 answers

databricks cli to update job shcedule

I have configured the Databricks cli locally and able to connect to the Azure Databricks cluster. Link reference used for my workstation - git Below command list the jobs successfully with the id $ databricks jobs list --profile dev Say if I…
Tim
  • 1,321
  • 1
  • 22
  • 47
3
votes
1 answer

Databricks CLI - delete folder if exists

Databricks throws an error when I try to delete a folder that doe not exist: databricks workspace delete -r /Shared/myfolder Error message: Error: b'{"error_code":"RESOURCE_DOES_NOT_EXIST","message":"Path (/Shared/myfolder) doesn\'t exist."}' So I…
kanimbla
  • 858
  • 1
  • 9
  • 23
2
votes
1 answer

"Invalid JSON given in the body of the request - expected a map" when using reset_job method

I am trying to change an existing job settings using the cli but when I invoke the reset_job method I am getting this error: Traceback (most recent call last): File "/home/vsts/work/1/s/S1.DataPlatform.DR/main.py", line 78, in …
Georgi Koemdzhiev
  • 11,421
  • 18
  • 62
  • 126
2
votes
1 answer

Databricks DBX named parameters through job

I am trying to implement this(where I don't have variables in the conf file but passed it as named arguments) mentioned here. When running in local mode and a python debugger, I can easily pass this as: Fundingsobj =…
Saugat Mukherjee
  • 778
  • 8
  • 32
1
2 3 4 5