3

I would like to create a databricks job where the owner is set to a service principal. Main reason is that if people leave company there is nothing to change and no job breaks due to failing PATs or similar. The problem is when doing so, that the job always fails to access the code which is set up as remote repo to azure devops. The problem and error message indicates that the service principal cannot clone the git repository.

How can i make this work?

My understanding was that I could use git-credentials api from databricks to let the service principal use one of my PAT tokens to access git with that. Did not work for me. https://docs.databricks.com/dev-tools/api/latest/gitcredentials.html#operation/create-git-credential

I know a workaround which is to build wheel and publish that, so that "checking out code" is not done by job but by CI/CD beforehand. But since remote repos is very simple and easy we would like to use that feature with service principals as well.

Did anyone use that and make it work? Maybe there is also the option to allow the service principal to access azure devops itself, but i was told that this is not possible yet.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
thompson
  • 179
  • 1
  • 12

1 Answers1

3

You're on correct way of doing it, maybe just miss some steps. The workflow is following:

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • It worked. My mistake was that when using Postman tool for testpurposes to set ADO PAT via GIT credentials API, for some reason i do not understand, the my AAD token was still used instead of the service principal AAD token. I tumbled over it when i wondered why AAD token was still valid after 2 hours of trying although Service Principal token was only valid for 1 hour....stupid me. But thx finally it works. You gave me hope that i was on right track and just missing something somewhere. – thompson Apr 03 '23 at 20:33
  • This explains a lot, but it still makes no sense to me, why 1. The Databricks UI makes no reference of this 2. The REPO documentation doesn't mentionen the requirement of Git credentials 3. The Microsoft CI/CD documentation neither mentions this requirement – Thomas Williams Aug 08 '23 at 12:04