I have a REST API in which out client sends the url of the image on their server. We then make a get request to their server and upload it to our s3 server. Now one of the clients sends the image url having https rather than http. When we try to fetch the image it fails randomly giving the error:-
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
Now the thing is I don't get this error for every call I make to the client. I wrote a script to check the same and found out that out of 50 requests I get this error for around 12-15 calls(Python 2.7.12 & reqeusts 2.4.3). First I thought this was error on the client end but on downgrading python from 2.7.13 to 2.7.10 I never get this error. I have read about this issue and all of the solutions point to the fact that I need to add path to CA BUNDLE, as stated here, when making the request. But if there was issue with the certificate then I must get error on 2.7.10 too & I should get the error everytime, not 10-15 times for 50 requests. There was this issue on github, that pointed to the same type of issue that I am facing but they are getting error for every request, not randomly.
Things that I have done so far:
- Downgraded certifi version to 2015.04.28 - no effect
- Changed the code from request.get(url) to request.get(url, verifi=ceritfi.old_where()) - no effect
- Downgraded python to 2.7.10 as told earlier - works
[UPDATE]:- I wrote a php script and made curl calls to multiple https endpoints of our client. It gave no error. Furthermore, I changed my Python code to use urllib3 instead of requests and downgraded my ceritifi to 2015.04.28, this setting also didn't give any error. Looks like something's up in the library being used by requests.
