Google Cloud SDK throws Reachability Check failed error

Question

I am trying to use gcloud init command to authenticate my account on mac os and it throws me error related to SSL Certificate. I am using python 3.7 and latest version of google clound sdk.

> gcloud info --run-diagnostics

Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
ERROR: Reachability Check failed.
    Cannot reach https://cloudresourcemanager.googleapis.com/v1beta1/projects with httplib2 (SSLCertVerificationError)
    Cannot reach https://www.googleapis.com/auth/cloud-platform with httplib2 (SSLCertVerificationError)
    Cannot reach https://cloudresourcemanager.googleapis.com/v1beta1/projects with requests (SSLError)
    Cannot reach https://www.googleapis.com/auth/cloud-platform with requests (SSLError)
Network connection problems may be due to proxy or firewall settings.

I am not behind any corporate proxy.

Update: Output for gcloud info --run-diagnostics --verbosity debug

DEBUG: Running [gcloud.info] with arguments: [--run-diagnostics: "True", --verbosity: "debug"]
Network diagnostic detects and fixes local network connection issues.
Checking network connection...⠶DEBUG: Starting new HTTPS connection (1): accounts.google.com:443
Checking network connection...⠧DEBUG: https://accounts.google.com:443 "GET / HTTP/1.1" 302 338
Checking network connection...⠏DEBUG: https://accounts.google.com:443 "GET /ServiceLogin?passive=1209600&continue=https%3A%2F%2Faccounts.google.com%2F&followup=https%3A%2F%2Faccounts.google.com%2F HTTP/1.1" 200 None
Checking network connection...⠛DEBUG: Starting new HTTPS connection (1): cloudresourcemanager.googleapis.com:443
DEBUG: Starting new HTTPS connection (1): www.googleapis.com:443
Checking network connection...⠹DEBUG: Starting new HTTPS connection (1): dl.google.com:443
Checking network connection...⠼DEBUG: https://dl.google.com:443 "GET /dl/cloudsdk/channels/rapid/components-2.json HTTP/1.1" 200 144486
Checking network connection...done.
ERROR: Reachability Check failed.
    Cannot reach https://cloudresourcemanager.googleapis.com/v1beta1/projects with httplib2 (SSLCertVerificationError)
    Cannot reach https://www.googleapis.com/auth/cloud-platform with httplib2 (SSLCertVerificationError)
    Cannot reach https://cloudresourcemanager.googleapis.com/v1beta1/projects with requests (SSLError)
    Cannot reach https://www.googleapis.com/auth/cloud-platform with requests (SSLError)
Network connection problems may be due to proxy or firewall settings.

Output for gcloud info as requested by @JohnHanley

Python Location: [/Library/Frameworks/Python.framework/Versions/3.7/bin/python3]
Site Packages: [Disabled]

Installation Root: [/home/myname/google-cloud-sdk]
Installed Components:
  gsutil: [4.55]
  core: [2020.12.04]
  bq: [2.0.64]
System PATH: [/home/myname/google-cloud-sdk/bin:/usr/local/opt/openssl@1.1/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/home/myname/Desktop/Projects/google-cloud-sdk/bin:/usr/local/opt/openssl@1.1/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/home/myname/opt/anaconda3/bin:/home/myname/opt/anaconda3/condabin:/home/myname/Library/Python/3.7/bin:/home/myname/Library/Python/3.7/bin]
Python PATH: [/home/myname/google-cloud-sdk/lib/third_party:/home/myname/google-cloud-sdk/lib:/Library/Frameworks/Python.framework/Versions/3.7/lib/python37.zip:/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7:/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload]
Cloud SDK on PATH: [True]
Kubectl on PATH: [/usr/local/bin/kubectl]

WARNING: There are other instances of the Google Cloud Platform tools on your system PATH.
  /home/myname/Desktop/Projects/google-cloud-sdk/bin/dev_appserver.py
  /home/myname/Desktop/Projects/google-cloud-sdk/bin/docker-credential-gcloud
  /home/myname/Desktop/Projects/google-cloud-sdk/bin/endpointscfg.py
  /home/myname/Desktop/Projects/google-cloud-sdk/bin/gcloud
  /home/myname/Desktop/Projects/google-cloud-sdk/bin/git-credential-gcloud.sh
  /home/myname/Desktop/Projects/google-cloud-sdk/bin/java_dev_appserver.sh
  /home/myname/Desktop/Projects/google-cloud-sdk/bin/anthoscli
  /home/myname/Desktop/Projects/google-cloud-sdk/bin/gsutil
  /home/myname/Desktop/Projects/google-cloud-sdk/bin/bq

Installation Properties: [/home/myname/google-cloud-sdk/properties]
User Config Directory: [/home/myname/.config/gcloud]
Active Configuration Name: [abc]
Active Configuration Path: [/home/myname/.config/gcloud/configurations/config_abc]

Account: [None]
Project: [None]

Current Properties:
  [core]
    disable_usage_reporting: [False]

Logs Directory: [/home/myname/.config/gcloud/logs]
Last Log File: [/home/myname/.config/gcloud/logs/2020.12.08/20.07.56.822405.log]

git: [git version 2.24.3 (Apple Git-128)]
ssh: [OpenSSH_8.1p1, LibreSSL 2.7.3]

Which OS version? Is it patched and updated? Are you behind a proxy? Your question needs more details. Post the output from `gcloud info` in your question. — John Hanley, Dec 09 '20 at 00:46
Need to see output of command gcloud info --run-diagnostics --verbosity debug — Mahboob, Dec 09 '20 at 00:59
@Mahboob I just updated the question with output for `gcloud info --run-diagnostics --verbosity debug` — Rushabh Patel, Dec 09 '20 at 01:09
@JohnHanley No, I am not behind proxy. It was working perfectly fine until I updated some python libraries and I am not sure which library messes it up. — Rushabh Patel, Dec 09 '20 at 01:10
That is your problem. The CLI uses its own version of Python and libraries. You have messed something up so that the CLI is using your installation of Python. You did not post the output I requested, so I can only make assumptions. — John Hanley, Dec 09 '20 at 01:12
@JohnHanley I just updated my question with `gcloud info` output. — Rushabh Patel, Dec 09 '20 at 01:17

John Hanley · Accepted Answer · 2020-12-09T02:03:24.617

You have a messed up system.

Look at the Python Location. Compare with Python PATH and PATH.
You have multiple Python and CLI installations in your PATH.
Your PATH mixes both 3.7 and 3.8 which is a recipe for disaster.
You have part of your installation relative to the Desktop.

My advice:

Remove all CLI installation.
Remove old Python installations - consider removing all versions and install fresh (after the steps below and rebooting) as I believe your Python libraries are now messed up.
Remove all Google CLI installations (gcloud).
Cleanup the path and remove directories that you removed.
Make sure remnants are not left behind.
Reboot.
Install the Google CLI fresh.

Tip. Do not install relative to you home directory for Python or the CLI.

Removing Python installations on macOS.

This is a summary. The exact steps depend on your setup. In essence you want to remove Python and the CLI from the PATH variable, the environment and the PYTHON varaibles. Also delete them from disk so that you can complete fresh installations.

The OS comes with its own installation. That is located in /Library/Frameworks/Python.framework. You have more than one installation repeated more than once. Read this article to remove those installations: https://www.macupdate.com/app/mac/5880/python/uninstall
I recommend modifying the PATH to remove the 3.7 and 3.8 versions and modify your environment to remove those references.
Remove /Library/Frameworks/Python.framework/Versions/3.7/lib/python37.zip from the PATH.
Delete the directory /home/myname/Library/Python/3.7/bin and contents.
Delete the directory /home/myname/Library/Python/3.8/bin and contents.
Read this link for removing the Google SDK https://cloud.google.com/sdk/docs/uninstall-cloud-sdk
Delete the environment variables related to PYTHON such as PYTHONPATH.
Delete anything starting with /home/myname/google-cloud-sdk from the PATH. Also delete the directory and contents.

Note: You also have Anaconda installed. I recommend removing that as it also has its own Python version.

Would you suggest any specific way to remove python installations? — Rushabh Patel, Dec 09 '20 at 01:47
@RushabhPatel - I updated my answer to include uninstall information. — John Hanley, Dec 09 '20 at 02:03
@RushabhPatel Post the new output from `gcloud info`. If you completely removed the Python installations, the SDK CLI installations, Anaconda, removed their respective directories, and cleaned the PATH and ENVIRONMENT variables, you will not have a problem. You skipped or left something behind. — John Hanley, Dec 09 '20 at 03:18

Ismail · Answer 2 · 2020-12-09T01:32:20.133

1

If your instance is configured to use SSL, go to the Lux Instances page in the Cloud Console and open the instance. Open its Connections page and make sure that your server certificate is valid. If it has expired, you must add a new certificate and rotate to it.

As per this previous thread, you would need to upgrade your certificate by running the following command:

pip3 install --upgrade certifi

Give it a try and let me know if you still need any help.

edited Dec 09 '20 at 01:32

answered Dec 09 '20 at 00:58

Ismail

1,068
1
6
11

1

Can you try to explain in more detail? Google does not setup SSL for VM instances. There is no Connections page. How is a VM's SSL certificate related to a problem with macOS? – John Hanley Dec 09 '20 at 01:04
You did not add detail for what I asked. Please reread my comment as I am curious what you are referring to. I understand the update (pip3 install). He does have a certificate problem but it is caused by his installation being broken. – John Hanley Dec 09 '20 at 01:38
@JohnHanley my answer is based on an internal doc that matched the error. I will try to further clarify later. – Ismail Dec 09 '20 at 20:43
I am not disagreeing with you on the pip install. If his Python installation is broken, Band-Aids might work. Eventually everything will break. Example, he updated his Python libraries and everything fell apart. Once the Python and Cloud SDK is setup properly, then updating modules is appropriate. – John Hanley Dec 09 '20 at 21:03
However, I am asking your to clarify this part of your answer as these pages/features do not exist and have nothing to do with macOS: `go to the Lux Instances page in the Cloud Console and open the instance. Open its Connections page` – John Hanley Dec 09 '20 at 21:03

score 0 · Answer 3 · answered Jun 09 '23 at 09:39

$ (security find-certificate -a -p ls /System/Library/Keychains/SystemRootCertificates.keychain && security find-certificate -a -p ls /Library/Keychains/System.keychain) > $HOME/.mac-ca-roots

$ export REQUESTS_CA_BUNDLE="$HOME/.mac-ca-roots"

$ gcloud config set core/custom_ca_certs_file $HOME/.mac-ca-roots

score 0 · Answer 4 · answered Aug 14 '23 at 20:42

I was confronted with this error as well, however instead of SSL errors I got timed out messages.

I had to disable IPv6. Like this for example (Ubuntu):

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1

Google Cloud SDK throws Reachability Check failed error

4 Answers4

Linked