9

I have deployed a Python 3.7 function in Google Cloud. I need to get the project-id through code to find out where it is deployed.

I write a small Python 3.7 script and test it through Google shell command line

import urllib
import urllib.request
url="http://metadata.google.internal/computeMetadata/v1/project/project-id"
x=urllib.request.urlopen(url)
with x as response:
 x.read()

Unfortunately this gives me only b'' as response. I am not getting the project id though I have set it using

gcloud config set project my-project

I am new to Google Cloud and Python.

Issue 2

This is an additional issue below:

In my local system I have installed gcloud and if I run the above python3.7 script from there:

x=urllib.request.urlopen(url) #this line

I am getting this exception from the line above:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/urllib/request.py", line 1350, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1240, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1286, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1235, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 1006, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 946, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/http/client.py", line 917, in connect
    self.sock = self._create_connection(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socket.py", line 787, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

The below changes as suggested by the answer also gives me empty string:

enter image description here

halfer
  • 19,824
  • 17
  • 99
  • 186
user1403505
  • 895
  • 2
  • 19
  • 42

3 Answers3

15

In the Python 3.7 runtime, you can get the project ID via an environment variable:

import os
project_id = os.environ['GCP_PROJECT']

In future runtimes, this environment variable will be unavailable, and you'll need to get the project ID from the metadata server:

import urllib.request
url = "http://metadata.google.internal/computeMetadata/v1/project/project-id"
req = urllib.request.Request(url)
req.add_header("Metadata-Flavor", "Google")
project_id = urllib.request.urlopen(req).read().decode()

Running this locally will produce an error because your local machine can't resolve the http://metadata.google.internal URL -- this is only available to deployed functions.

Dustin Ingram
  • 20,502
  • 7
  • 59
  • 82
  • Hi Thanks for this, I ran this at cloud shell, but the project_id is coming as empty string " " >>> import urllib.request >>> url = "http://metadata.google.internal/computeMetadata/v1/project/project-id" >>> req = urllib.request.Request(url) >>> req.add_header("Metadata-Flavor", "Google") >>> project_id = urllib.request.urlopen(req).read().decode() >>> project_id '' >>> print(project_id) >>> – user1403505 Dec 02 '20 at 05:47
  • 1
    As mentioned by @Dustin Ingram, it will not work locally or in Cloud Shell, it is only available for deployed functions. I add an another answer giving an alternative, – Nibrass H Dec 02 '20 at 16:21
  • 1
    And please not that you are getting an empty string because you are not replacing projec-id from the url to one's of yours. – Nibrass H Dec 02 '20 at 16:36
  • @Nibrass H yes i understood thank you, but my goal is to find the project id which i have set to currently and not the default project id, so will this metadata-api find the current project id without me giving it in the url? metadata.google.internal/computeMetadata/v1/project/ – user1403505 Dec 02 '20 at 17:02
  • It's correct that this won't work in Cloud Shell, but you don't need to modify the URL. This is an endpoint for determining the current project ID. – Dustin Ingram Dec 02 '20 at 19:44
  • Thanks @DustinIngram using this metadata url if a user wants to get the project id dynamically during function execution, is it not possible, using thie api url as the api requires a project-id parameter by itself – user1403505 Dec 03 '20 at 07:48
  • No, it doesn't. The example works exactly as shown. – Dustin Ingram Dec 03 '20 at 13:19
  • not to zombie this comment though @DustinIngram but was the reasoning behind why this change was made discussed? also are older in 3.7 before going to become RESERVED words? – StanleyZheng May 24 '22 at 17:29
3

In Cloud Shell, the DEVSHELL_PROJECT_ID environment variable contains your project ID.

So you can run the following command in order to get your project ID with Python.

import os
USER = os.getenv('DEVSHELL_PROJECT_ID')
print(USER)

And if you want to get your project ID in a Cloud Function, you can get it from the service account credentials.json which contains the project_id:

# If this is running in a cloud function, then GCP_PROJECT should be defined
if 'GCP_PROJECT' in os.environ:
    project_id = os.environ['GCP_PROJECT']
    
# else if this is running locally then GOOGLE_APPLICATION_CREDENTIALS should be defined
elif 'GOOGLE_APPLICATION_CREDENTIALS' in os.environ:
    with open(os.environ['GOOGLE_APPLICATION_CREDENTIALS'], 'r') as fp:
        credentials = json.load(fp)
    project_id = credentials['project_id']
else:
    raise Exception('Failed to determine project_id')
Nibrass H
  • 2,403
  • 1
  • 8
  • 14
  • Excellent and detailed, just a clarification your code works in cloud shell , also if i understand, if i dont set any environment id like GCP_Project is it not possible for me to get the project id of the cloud function where it is deployed? and I also tested GOOGLE_APPLICATION_CREDENTIALS (your elif part) my google_application_credentials is for project A, so if change the project to project B it still shows for project A only, as it is for that service account, so I change it to any other service account, is there a way? – user1403505 Dec 02 '20 at 17:00
  • 1
    Using `os.environ['GCP_PROJECT']` is deprecated and won't be available in subsequent runtimes. Users should use the metadata server instead. – Dustin Ingram Dec 02 '20 at 19:47
0

Google has released an official metadata library for Go, which can be found at cloud.google.com/go/compute/metadata. To get the project ID, you can simply use the ProjectID() function. For more details, please visit https://pkg.go.dev/cloud.google.com/go/compute/metadata#ProjectID.

Woojong Koh
  • 1