0

I am trying to scrape the following page: https://www.fanduel.com/games/57764/contests/57764-245891325/scoring

The following code that uses urlopen() produced a certificate verify failed error:

url = 'https://www.fanduel.com/games/57764/contests/57764-245891325/scoring'
html = urlopen(url)
soup = BeautifulSoup(html, 'html.parser')

Error: urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1123)>

I also tried using the following code, but I've received a <response [403]>

html = f'https://www.fanduel.com/games/57764/contests/57764-245891325/scoring'
r = get(html, verify=False)

Would greatly appreciate any suggestions for scraping this site, whether that's a code update or a recommendation to use a different web-scraping package. Thanks!

UPDATE PER Maxlovesairandteslas Response:

I'm now encountering a new error. Within the response it says: access to this page has been denied. I updated my code as such, so I'm assuming that I'm at least getting through to the appropriate page and now being denied:

with requests.Session() as s: 
   p = s.post("fanduel.com/login", verify=False, data={"email": "","password": ""}) 
   base_page = s.get('fanduel.com/games/57764/contests/57764-245891325/scoring')#, headers=headers) 
   soup = BeautifulSoup(base_page.content, 'html.parser')
   print(soup.prettify())
Michael M
  • 57
  • 8
  • It seems that the website requires you to log in. See [How to “log in” to a website using Python's Requests module?](https://stackoverflow.com/questions/11892729/how-to-log-in-to-a-website-using-pythons-requests-module). – MendelG Apr 26 '21 at 22:17

1 Answers1

0

It seems like you need to log in to the site before you can play the game. Like what @MendelG said, try this: log in to website

Maxlovesairandteslas
  • 1
  • 6
  • This seems to be helping, but I'm now encountering a new error. Within the response it says: access to this page has been denied. I updated my code as such, so I'm assuming that I'm at least getting through to the appropriate page and now being denied: with requests.Session() as s: p = s.post("https://www.fanduel.com/login", verify=False, data={"email": "***","password": "***"}) base_page = s.get('https://www.fanduel.com/games/57764/contests/57764-245891325/scoring')#, headers=headers) soup = BeautifulSoup(base_page.content, 'html.parser') print(soup.prettify()) – Michael M Apr 26 '21 at 23:07