1

I am trying to get data from Google big query table using python. I dont have a service account access,but i have individual access to bigquery using gcloud. i have application default credentials Json file. I need to how to make a connection to bigquery usinG ADC.

code snippet:

from google.cloud import bigquery
conn=bigquery.Client()
query="select * from my_data.test1"
conn.query(query)

When i run above code snippet i am getting error saying:

NewConnectionError: <urllib3.connection.HttpsConnection object at 0x83dh46bdu640>: Failed to establish a new connection:[Error -2] Name or Service not known

Note: ENVIRONMENT Variable GOOGLE APPLICATION CREDENTIALS is not set and empty

Yogesh Chander
  • 43
  • 5

1 Answers1

2

Your script works for me because I authenticated using end user credentials from Google Cloud SDK, once you have the SDK installed you can simply run:

gcloud auth application-default login

The credentials from your json file are not being passed to the bigquery client, e.g.:

client = bigquery.Client(project=project, credentials=credentials)

to set that up you can follow these steps: https://cloud.google.com/bigquery/docs/authentication/end-user-installed

or this thread has some good details on setting the credentials environment variable: Setting GOOGLE_APPLICATION_CREDENTIALS for BigQuery Python CLI

pmo511
  • 569
  • 3
  • 9
  • Thanks for reply, i have run this below command 'gcloud auth application-default login' after this it created a application_default_credentials.json file in my home directory of on prim server. should i use that json file or i should generate a json file from the steps in the link you have shared ? – Yogesh Chander Jul 16 '22 at 18:33
  • If you're using the terminal within the IDE VS Code it should work now. However, if you're using Jupyter notebooks then yes, you'll still have to load your credentials form the file as per this step: https://cloud.google.com/bigquery/docs/authentication/end-user-installed#authenticate-and-call-the-api – pmo511 Jul 19 '22 at 20:09
  • I am using jupyter notebook only. The thing is i have been provided with application_defalut_crdentials.json only. So according to u i need also client_secret.json file for connecting to bigquery. If there is any other way using ADC.json file we can do something let me know. Thanks in advance – Yogesh Chander Jul 20 '22 at 13:01
  • I run Jupyter notebooks via a virtual environment and don't need to pass credentials and just login using `gcloud auth application-default login`. From memory, for a standalone jupyter setup maybe need to pass your client_secret.json as you mentioned above... I highly recommend trying a virtual environment though, here are the steps: 1. pip install virtualenv 2. virtualenv .venv 3. source .venv/bin/activate 4. pip install -r requirements.txt # inc dependencies eg google-cloud-bigquery==2.24.1 5. ipython kernel install --user --name=.venv 6. jupyter notebook (pretty cool, give it a go) – pmo511 Jul 20 '22 at 19:06