Databricks REST API call for updating branch error : User Settings > Git Integration to set up an Azure DevOps personal access token

Question

I am getting below error for updating the repo to a different branch using databricks rest api as mentioned at https://docs.databricks.com/dev-tools/api/latest/repos.html#operation/update-repo .

I have authenticated using service principal and generated dbrks_bearer_token and dbrks_management_token.

Please find below code for same :-

import requests
import os
import json

TOKEN_REQ_BODY = {
    'grant_type': 'client_credentials',
    'client_id':  'client_id',
    'client_secret':  'client_secret'
}
TOKEN_BASE_URL = 'https://login.microsoftonline.com/' +  'tenant_id' + '/oauth2/token'
TOKEN_REQ_HEADERS = {'Content-Type': 'application/x-www-form-urlencoded'}


def dbrks_management_token():
    TOKEN_REQ_BODY['resource'] = 'https://management.core.windows.net/'
    response = requests.get(TOKEN_BASE_URL, headers=TOKEN_REQ_HEADERS, data=TOKEN_REQ_BODY)
    if response.status_code == 200:
        print(response.status_code)
        return response.json()['access_token']
    else:
        raise Exception(response.text)
        return response.json()['access_token']

def dbrks_bearer_token():
    TOKEN_REQ_BODY['resource'] = '2ff814a6-3304-4ab8-85cb-cd0e6f879c1d'
    response = requests.get(TOKEN_BASE_URL, headers=TOKEN_REQ_HEADERS, data=TOKEN_REQ_BODY)
    if response.status_code == 200:
        print(response.status_code)
    else:
        raise Exception(response.text)
    return response.json()['access_token']

DBRKS_BEARER_TOKEN = dbrks_bearer_token()
DBRKS_MANAGEMENT_TOKEN = dbrks_management_token()


DBRKS_REQ_HEADERS = {
    'Authorization': 'Bearer ' + DBRKS_BEARER_TOKEN,
    'X-Databricks-Azure-Workspace-Resource-Id':
        '/subscriptions/susbcriptionid' +
        '/resourceGroups/rg-dev/providers/Microsoft.Databricks/workspaces/dbr-dev',
    'X-Databricks-Azure-SP-Management-Token':  DBRKS_MANAGEMENT_TOKEN }

DBRKS_CLUSTER_ID = {'cluster_id': 'cluster_id'}

def update_repo():
    DBRKS_START_ENDPOINT = 'api/2.0/repos/0328767704612345'
    postjson = """{
     "branch": "main"
  }"""

    response = requests.patch("https://adb-1234582271731234.0.azuredatabricks.net/"
                              + DBRKS_START_ENDPOINT,
                              headers=DBRKS_REQ_HEADERS,
                              json=json.loads(postjson))
    print(response.status_code)
    if response.status_code != 200:
        raise Exception(response.text)

    os.environ["DBRKS_CLUSTER_ID"] = response.json()["cluster_id"]
    print(response.content)

update_repo()

I am getting below error:-

Traceback (most recent call last):
  File "C:/Users/IdeaProjects/DBCluster/DB_rest_api.py", line 109, in <module>
    update_repo()
  File "C:/Users/IdeaProjects/DBCluster/DB_rest_api.py", line 104, in update_repo
    raise Exception(response.text)
Exception: {"error_code":"PERMISSION_DENIED","message":"Missing Git provider credentials. Go to User Settings > Git Integration to add your personal access token."}
403

Can someone please let me know if i need to anything explicitly at azure devops git configuration level as well?

Many thanks..!!

score 1 · Answer 1 · answered May 15 '22 at 11:39

1

To do any operations with Databricks Repos, you need to set a Git personal access token. Because you're using service principal, you can't set that personal access token via UI, but you can perform setting of that Git token using recently implemented Git Credentials REST API - just do that once for your service principal (or when Git PAT expires), and then Repos operations will work.

answered May 15 '22 at 11:39

Alex Ott

80,552
8
87
132

I am still confused how to do it using Git Credentials REST API but I have done it manually using PAT and DB Token. Is it possible for you to share any links/blog where it has been done using SPN?? Many Thanks..!! – rpshgupta May 17 '22 at 16:57
it's done similarly to what you do in your code - generate bearer & management tokens and call corresponding Git Credentials API – Alex Ott May 17 '22 at 17:56
Just commenting to add that I have succesfully used git-credentials API to add the bearer token generated with the msal library, but trying to use the Repos API afterwards results in the same error message as in OP. – Bjarne Thorsted May 19 '22 at 23:16
For DevOps you need real PAT, not AAD token – Alex Ott May 20 '22 at 05:09
How can I get a real PAT for a service principal/app registration? Is that a premium feature or can I somehow use a PAT generated for a user? – Bjarne Thorsted May 20 '22 at 05:11
it's explained in this answer: https://stackoverflow.com/questions/72256036/azure-databricks-api-cannot-add-repos-using-service-principal-and-api-calls/72273439#72273439 – Alex Ott May 20 '22 at 06:03

Databricks REST API call for updating branch error : User Settings > Git Integration to set up an Azure DevOps personal access token

1 Answers1