1

I am getting the following 403 Forbidden response when trying to access a site from within my Python app. I am NOT being challenged by CloudFlare with any Captcha as far as I can tell, as is the case in a lot of other people’s similar questions, it’s asking me to enable cookies. The website returns 200 OK if I try via CURL or via any browser, so it’s not IP restrictions, it’s just my Python Request it doesn’t like. I have tried various combinations of User-Agent to no avail, tried http, https and nothing at all before the target URL, and I’ve mimicked exactly what the browser Network Inspector shows in the requests header from a successful regular browser GET.

Here’s the error in the http response: (status 403)

Please enable cookies. Error 1020 Ray ID: 69c89e49895c40d7 • 2021-10-11 14:01:04 UTC Access denied What happened? This website is using a security service to protect itself from online attacks. Cloudflare Ray ID: 69c89e49895c40d7 • Your IP: x.x.x.x • Performance & security by Cloudflare Please enable cookies.

Here’s my Python: ''' r = requests.get( “www.oddschecker.com”, headers={ "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0", "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8", "Accept-Language": "en-GB,en;q=0.5", "method": "GET", "content-type": "text/plain", "accept-encoding": "gzip, deflate, br", "Connection": "keep-alive", "scheme": "https", "Upgrade-Insecure-Requests": "1", "Cache-Control": "max-age=0", "Host": "www.oddschecker.com", "TE":"Trailers" },)

'''

Questions:

  • How does CloudFlare know that I need to enable cookies or that I’m not a regular browser just from my Python request? I send the request and get an immediate 403 back. The request is exactly the same as if I use a browser. It’s almost as though there’s some traffic going on that network inspector doesn’t show between my request and the 403. I used Fiddler too, and that just shows the same: GET request, immediate 403 response.

  • How DO I enable cookies within Python?

andyguest13
  • 23
  • 5

1 Answers1

1

The Python Requests library has support for adding a cookie dictionary: https://stackoverflow.com/a/7164897/13343799

You can see your cookies key-value pairs by (in Chrome) clicking F12 to open Developer Options -> Application tab -> Cookies -> select a cookie and see the Cookie Value below it. Key is before the =, value is after.

Nate Harris
  • 36
  • 4
  • 1
    Thanks for the info, that's helpful on cookies - but not sure how that helps me with my problem. To restate: my python seems to be sending the same Request as Firefox does. But I get an immediate 403, whereas Firefox gets a 200 response (admittedly with a bunch of set-cookies in it). So I don;t even get the opportunity to execute any code that does anything re cookies. – andyguest13 Oct 14 '21 at 17:42