0

This is my code snippet and i get the SSL error when i run this code.I tried giving a delay for the code and working on it but i still get the same error.

import os
import requests
from urllib.parse import urljoin
from bs4 import BeautifulSoup
url = "http://bbmp.gov.in/en/covid19bulletins"
#If there is no such folder, the script will create one automatically
folder_location = r'C:\Users\maria.fh\Documents\Automatically downloaded files'
if not os.path.exists(folder_location):os.mkdir(folder_location)

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

session = requests.Session()
retry = Retry(connect=3, backoff_factor=0.8)
adapter = HTTPAdapter(max_retries=retry)
session.mount('http://', adapter)
session.mount('https://', adapter)

session.get(url)
response = requests.get(url)
soup= BeautifulSoup(response.text, "html.parser")     
for link in soup.select("a[href$='.pdf']"):
    filename = os.path.join(folder_location,link['href'].split('/')[-1])
    with open(filename, 'wb') as f:
        f.write(requests.get(urljoin(url,link['href'])).content)

this is the SSL exception raised:

requests.exceptions.SSLError: HTTPSConnectionPool(host='dl.bbmpgov.in', port=443): Max retries exceeded with url: /covid/Covid_Bengaluru_26June_2020%20Bulletin-95%20English.pdf (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)')))
mrxra
  • 852
  • 1
  • 6
  • 9
maria francis
  • 3
  • 1

1 Answers1

0

your client (requests) fails to validate the server's SSL certificate. you could add verify=False to your call: f.write(requests.get(urljoin(url,link['href']), verify=False).content). this is however NOT recommended. a quick search on stackoverflow regarding the generated error message will guide you to a lot of useful threads which propose real solutions, e.g. https://stackoverflow.com/a/57466119/12693728

mrxra
  • 852
  • 1
  • 6
  • 9
  • Thanks a lot that worked, i did the same with requests.get('http://bbmp.gov.in/en/covid19bulletins', verify=False) but it showed me the same error...but this one worked – maria francis Jul 10 '20 at 09:59