1

I have a simple call to hit an amazon product page:

response = requests.get('https://www.amazon.com/dp/...')
print(response.url)
print(response.history

When I physically navigate to the page in my browser (chrome), I get redirected to a new product page that's not the one I searched.

My response however, doesn't seem to pick up the redirects and the history is empty and the response url is the same as what I requested. I have tested this on other URLs that I know redirect, but those work fine. It's only when dealing with Amazon.com/dp/productId pages.

I want to capture that redirect URL, but am unable to. Any idea what might be going wrong here?

Edit: Some additional insights. It appears that the redirect occurs after it navigates to the site. So something in the page javascript I believe might be what's triggering it to redirect, so my basic request.get() never picks it up.

Jet.B.Pope
  • 642
  • 1
  • 5
  • 25
  • can you provide the exact sample URL. I want to test and see myself. thx – Fuat Ulugay May 24 '23 at 18:14
  • 1
    For example, I tested this one: https://www.amazon.com/dp/B0B9BGQKXJ and there is no redirect in chrome or in python. So if you provide your example I can check. – Fuat Ulugay May 24 '23 at 18:28
  • 1
    Same, I'm having difficulty reproducing this issue. Please provide a URL that you're having issues with and the URL you expect it to redirect to so that we can reproduce this issue and help you. https://stackoverflow.com/help/minimal-reproducible-example – Andria May 24 '23 at 21:03
  • Can you share the output from `response.history`? You can use the following code to check the status code and URLs: for res in response.history: print(res.status_code, res.url) The status codes for redirects take the form 30x. The type of redirect dictates the last value. For instance, 301 indicates a permanent redirect, while 302 indicates a temporary redirect, according to this [source](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections). You can also check these related answers:
    [https://stackoverflow.com/a/55662431/21951045](https://stackoverflow.com/a/55662431/21951045)
     – Writa H May 31 '23 at 05:42

0 Answers0