0

I am running into a similar problem as the following one (unable to get response from nasdaq):

No response from `request.get()` for NASDAQ webpage

After adding user-agent, it works on my computer. However, it still does not work when I run this on AWS debian instance. I verified the following:

  1. Manually open the endpoint in firefox works
  2. Verified the user-agent is the same as the firefox one
  3. Requesting a different site (google.com) works

At this point, it seems that the user-agent I set was not working for some reason. Has anyone seen this and can provide some help?

Thanks.

Ye

knightyangpku
  • 75
  • 1
  • 4
  • If NASDAQ are blocking requests from non-browsers, it's likely they're also blocking requests from known cloudservice providers like Google, AWS and Azure. This would explain why your home computer works but your AWS one does not. I'd guess that scraping their site breaches their Terms of Service, and that there is an API that you should use, and probably have to pay for. – Tangentially Perpendicular May 14 '23 at 23:47

0 Answers0