Stack Overflow
Stack Overflow

Questions tagged [databricks-rest-api]

For questions around Databricks REST APIs that are used for automation of Databricks.

Databricks REST APIs are used to work with Databricks from the code - CI/CD, deployment, ... These APIs are the base for all other tools - Databricks CLI, Databricks Terraform Provider, etc.

31 questions
3
votes
1 answer

How to run a Azure DataBricks Notebook and get it's result via Rest API

I have a use case where I need to run a set of notebooks developed in Azure Databricks (that performs several queries and calculations), but the end user (non-technical) needs only the final result, displayed in a web page. I searched the Azure…
Lucas Ponzo
  • 104
  • 1
  • 6
3
votes
1 answer

How to write Notebook to Databricks using API?

I am able to write data files to Databricks using the APIs documented here: https://docs.databricks.com/dev-tools/api/latest/index.html https://docs.databricks.com/dev-tools/api/latest/dbfs.html I have several Notebooks I would like to write to a…
John
  • 3,458
  • 4
  • 33
  • 54
2
votes
1 answer

How to edit permissions of Databricks queries with REST API?

I am trying to give CAN_MANAGE permission to other users for a given set of SQL queries on Azure Databricks. Browsing Databricks REST API reference, under the Databricks SQL > ACL / Permissions section there are both Get object ACL and Set object…
followingell
  • 413
  • 6
  • 12
2
votes
1 answer

"Invalid JSON given in the body of the request - expected a map" when using reset_job method

I am trying to change an existing job settings using the cli but when I invoke the reset_job method I am getting this error: Traceback (most recent call last): File "/home/vsts/work/1/s/S1.DataPlatform.DR/main.py", line 78, in …
Georgi Koemdzhiev
  • 11,421
  • 18
  • 62
  • 126
2
votes
1 answer

Databricks REST API call for updating branch error : User Settings > Git Integration to set up an Azure DevOps personal access token

I am getting below error for updating the repo to a different branch using databricks rest api as mentioned at https://docs.databricks.com/dev-tools/api/latest/repos.html#operation/update-repo . I have authenticated using service principal and…
rpshgupta
  • 135
  • 1
  • 8
2
votes
1 answer

Execute git pull on databricks notebook using CLI and/or API

Using Databricks Repos, you can add a git repo to Databricks and execute git actions such as git pull. This is done by clicking on the branch name in the top left, and clicking the button saying "Pull". I would like to do this without clicking on…
Haterind
  • 1,095
  • 1
  • 8
  • 16
1
vote
2 answers

How to call databricks rest API to list jobs run

I am currently developing a Python script to retrieve a comprehensive list of all the jobs that were executed yesterday. However, I'm encountering an issue with the script's pagination mechanism using tokens. Despite my attempts to loop through the…
Shahid Haider
  • 43
  • 5
1
vote
0 answers

how to call databricks notebook from python with rest api

I want to create a python notebook on my desktop that pass an input to another notebook in databricks, and then return the output of the databricks notebook. For example, my local python file will pass a string into a databricks notebook, which will…
J. Doe
  • 165
  • 5
  • 16
1
vote
2 answers

Creating REST api in databricks which would display delta tables information to the user?

I would like create an api endpoint in azure databricks which would publish delta table data to user.All i found by searching there is no restapi to expose delta table content.Is it True?
Saswat Ray
  • 141
  • 3
  • 14
1
vote
1 answer

Databricks: Clear Contents of All Cells in All Notebooks

We have a need to ensure that all data of a given type have been removed from our system. This includes any cell output that might exist in a Databricks Notebook. Is there a way to clear the cell outputs from all cells in all Notebooks starting from…
John
  • 3,458
  • 4
  • 33
  • 54
1
vote
1 answer

Databricks API - Instance Pool - How to update an existing job to use instance pool instead?

I am trying to update a batch of jobs to use some instance pools with the databricks api and when I try to use the update endpoint, the job just does not update. It says it executed without errors, but when I check the job, it was not updated. What…
MaxAirheart
  • 33
  • 5
1
vote
2 answers

Can we set task wise parameters using Databricks Jobs API "run-now"

I have a job with multiple tasks like Task1 -> Task2. I am trying to call the job using api "run now". Task details are below Task1 - It executes a Note Book with some input parameters Task2 - It executes a Note Book with some input parameters So,…
Rudra
  • 138
  • 6
1
vote
1 answer

Databricks API - Instance Pool - How to create with photon enabled?

I am trying to create an instance pool using databricks api and I need the photon enabled, in the documentation I could not find the parameter to enable it, does anyone know how to do it? I used this…
MaxAirheart
  • 33
  • 5
1
vote
2 answers

Is there any difference between the Job data returned from Databricks Jobs API 2.1 vs 2.0?

The main difference I see is that 2.1 requires paging results with a max page size of 25, while 2.0 does not support paging and will get you all of the results in one call. Is there any difference in the structure or content of the "Job" objects…
successhawk
  • 3,071
  • 3
  • 28
  • 44
1
vote
1 answer

Databricks Jobs Webhook Notification ID

I want to set a webhook notification for a job using the Jobs API. According to the documentation I need the notification ID, but where do I get that from? Is that the essentially the webhook configuration name? An optional list of notification IDs…
Robert Kossendey
  • 6,733
  • 2
  • 12
  • 42
1
2 3