0

I am trying to use requests module of Python3 to make an API call. I am using credentials that work when tried on same service from their web page. So credentials are verified.

Code that I am using:

#!/local/usr/bin/python3

import requests
url = "https://some.url.com/fol/something"
payload = "{'userName': \"user-name\", 'instanceName': \"instance-name\", 'password': \"user-password\", 'usersDomain': \"DOMAIN\"}"
headers = {'accept': 'application/json','content-type': 'application/json'}
response = requests.request("POST", url, data=payload, headers=headers, verify=False)
print(response)

Problem:

If I remove verify=False from line 7 I get ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749).

With verify=False (as in code above) I get following warning and response code is 401.

InsecureRequestWarning: Unverified HTTPS request is being made to host 'wifitracker.chartercom.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
<Response [401]>

What I have tried:

  1. Tried using same code from different Linux servers with different Python and requests module version.
  2. Tried ping {host} and nc -vz {host} 443 and both succeed. The API endpoint and Linux server from where I am trying to connect are both on the same network domain.
  3. Tried adding verify="/etc/ssl/certs/ca-bundle.crt" and verify="/etc/ssl/certs/ca-bundle.trust.crt" as well as verify="/etc/ssl/certs" and verify="/etc/ssl/certs/". All these options don't cause any error/warning but the response code is still 401.
  4. Used python module certifi and the found the installed certification authority budle and used it in the veryify= but I get ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)
  5. Following suggestion by Steffen I tried openssl s_client -connect some.url.com:443 and result is (hope it's not too much):
USR--> openssl s_client -connect some.url.com:443
CONNECTED(00000003)
depth=2 C = <My Country Code>, O = <My Organization Name>, OU = <My Organization Name> Internal Tr<My Country Code>t Network, OU = (c) 2017 <My Organization Name> - For authorized <My Country Code>e only, CN = <My Organization Name> Root Certification Authority
verify error:num=19:self signed certificate in certificate chain
---
Certificate chain
 0 s:/C=<My Country Code>/ST=<My State>/L=<My City>/O=<My Organization Name>/CN=some.url.com
   i:/DC=com/DC=url/DC=corp/CN=<My Organization Name> Issuing CA1
 1 s:/DC=com/DC=url/DC=corp/CN=<My Organization Name> Issuing CA1
   i:/C=<My Country Code>/O=<My Organization Name>/OU=<My Organization Name> Internal Tr<My Country Code>t Network/OU=(c) 2017 <My Organization Name> - For authorized <My Country Code>e only/CN=<My Organization Name> Root Certification Authority
 2 s:/C=<My Country Code>/O=<My Organization Name>/OU=<My Organization Name> Internal Tr<My Country Code>t Network/OU=(c) 2017 <My Organization Name> - For authorized <My Country Code>e only/CN=<My Organization Name> Root Certification Authority
   i:/C=<My Country Code>/O=<My Organization Name>/OU=<My Organization Name> Internal Tr<My Country Code>t Network/OU=(c) 2017 <My Organization Name> - For authorized <My Country Code>e only/CN=<My Organization Name> Root Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
<CHUNK OF TEXT>
-----END CERTIFICATE-----
subject=/C=<My Country Code>/ST=<My State>/L=<My City>/O=<My Organization Name>/CN=some.url.com
issuer=/DC=com/DC=url/DC=corp/CN=<My Organization Name> Issuing CA1
---
No client certificate CA names sent
Peer signing digest: SHA256
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 6834 bytes and written 433 bytes
---
New, TLSv1/SSLv3, Cipher is <SOME-ALHPA-NUMERIC-CHARACTERS>
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : <SOME-ALHPA-NUMERIC-CHARACTERS>
    Session-ID: <SOMELONGALHPANUMERIC64CHARACTERS>
    Session-ID-ctx:
    Master-Key: <SOMELONGALHPANUMERIC96CHARACTERS>
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: <epoch_time>
    Timeout   : 300 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
---

I tried finding options (from here on SO and other resources) and think I have tried those but obviously I am still missing something.

What else can I try to make a successful API call (status code 200)? Could it be that I need ca cert from the API provider? (It's a security related application)

300
  • 965
  • 1
  • 14
  • 53
  • 1
    There are many many similar questions here on SO, just search for the error. The problem can be caused by micsconfiguration on the server site like self-signed certificate or missing intermediate certificate, by SSL intercepting middleboxes like company firewalls, by a local trust store which is not current ... . Based on the given information it is unclear what exactly the problem is here and thus no recommendations can be done on what need to be done to fix the problem. – Steffen Ullrich Oct 04 '21 at 04:48
  • Thank you for your response and suggestion @Steffen Ullrich. I have tried searching here on SO and other sources and tried those options. So far I haven't been successful. But I am continuing on that path. Please let me know how I can provide more information that could help to find the problem or narrow it down. Appreciate your time and suggestions. – 300 Oct 04 '21 at 11:07
  • 1
    *"how I can provide more information"* - run `openssl s_client -connect {host]:443` and add as much from the output to your question as possible, but at least the shown *Certificate chain* and the *Verify return code*. – Steffen Ullrich Oct 04 '21 at 12:34
  • Thank you again Steffen. I have added output of `openssl s_client -connect {host]:443` in question. – 300 Oct 04 '21 at 15:31
  • 1
    Thanks. From the output it gets obvious that you are using an internal CA. This CA is not inside the normal trust store and certify does not provide it either. You explicitly need to trust this CA for the operation using the `verify` argument or with environment variables - see for example [How to get Python requests to trust a self signed SSL certificate?](https://stackoverflow.com/questions/30405867/how-to-get-python-requests-to-trust-a-self-signed-ssl-certificate). – Steffen Ullrich Oct 04 '21 at 16:01
  • Just wanted to share my progress and say thank you for your help so far: I did try to add path to the .crt file or directory but somehow I still get 401 status code. Following instructions from the article you shared I did download `domain.pem` and `domain-chain.pem` files by visiting the API URL from Firefox. Then tried providing path to these two files or the directory under which these are and still getting `ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)` After this error I do get another `Max retries exceeded with url:` – 300 Oct 04 '21 at 21:05
  • If you get a status code of 401 then the TLS handshake was already successful, since the 401 is from the HTTP dialog inside the TLS handshake. And the certificate you need to add as trusted are not the server certificate and chain, but the root certificate, i.e. `s:/C=/O=/OU= Internal Trt Network/OU=(c) 2017 - For authorized e only/CN= Root Certification Authority`. – Steffen Ullrich Oct 04 '21 at 21:23
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/237801/discussion-between-300-and-steffen-ullrich). – 300 Oct 04 '21 at 21:24

0 Answers0