4

I am attempting to make a connection to Databricks from my Mac(Mojave).

I did a pip install -U databricks-connect==5.5.*

I start a spark-shell but when I try to query in spark I get the following error

Caused by: com.databricks.service.SparkServiceConnectionException: Invalid token

To connect to a Databricks cluster, you must specify an API token.
API Token: The API token used to confirm your identity to Databricks
  - Learn more about API tokens here: https://docs.databricks.com/api/latest/authentication.html#generate-a-token
  - Get current value: spark.conf.get("spark.databricks.service.token")
  - Set via conf: spark.conf.set("spark.databricks.service.token", <your API token>)
  - Set via environment variable: export DATABRICKS_API_TOKEN=<your API token>

I then run spark.conf.get("spark.databricks.service.token") and I get back the token I created. We tried using this token on a windows machine and it works.

DataTx
  • 1,839
  • 3
  • 26
  • 49
  • Have you tried creating a new token and using it? – Cal Mar 03 '20 at 16:04
  • yes, I sure have. – DataTx Mar 03 '20 at 16:05
  • How do you usually feed the token to spark? Via conf file, environment vairable, or spark.conf.set? – Cal Mar 03 '20 at 16:06
  • I'm using dbconnect through conda. There is a databricks-connect configure command where the host name, cluster name, token etc are configured. – DataTx Mar 03 '20 at 16:07
  • I trust you have been following [this documentation](https://docs.databricks.com/dev-tools/databricks-connect.html) what does `databricks-connect test` return? – Cal Mar 03 '20 at 16:10
  • yes sir. I get the same error that is posted in the question. – DataTx Mar 03 '20 at 16:11
  • And when you re-run `databricks-connect configure` again it still provides a valid token? Did you uninstall before doing all that? – Cal Mar 03 '20 at 16:27
  • Not during this round but I've done it before I posted the question. – DataTx Mar 03 '20 at 16:32
  • This seems to be a Mac issue. I had a coworker create a new token and attempt to connect but it did not work. – DataTx Mar 03 '20 at 16:33
  • Hope you find a workaround, I had no issues with either Windows or Ubuntu. – Cal Mar 03 '20 at 16:43

1 Answers1

3

Run:

databricks-connect configure

Follow the steps on the command prompt. You will be asked to enter your Databricks Token, enter your token, follow remaining steps and it should be resolved.

PS: When you are asked to enter your Databricks Host: provide a generic hostname for the region followed by your org-id. For example:

https://westeurope.azuredatabricks.net/?o=xxxx instead of https://adb-xxxx.xy.azuredatabricks.net/?o=xxxx

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
pprasad009
  • 508
  • 6
  • 9