Not getting data from request in python

Asked Jul 08 '20 at 00:15

Active Jul 08 '20 at 00:24

Viewed 87 times

I was trying to fetch a product info from an online store. It doesn't require any kind of authentication. I can see product information from the browser. I've inspected in developer tools and found the request which fetches data from the URL. My python code is bellow

import requests

def coles():
    URL = "https://shop.coles.com.au/search/resources/store/20520/productview/bySeoUrlKeyword/mutti-tomato-passata-2349503p"
    PARAMS = {
        "catalogId": 12064
    }
    res = requests.get(URL,PARAMS)
    print(res.text)


#coles()
if __name__ == '__main__':
    coles()

But I'm getting this instead of the product info

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <link rel="shortcut icon" href="about:blank">
</head>
<body>
<script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/j.js"></script>
<script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/f.js"></script>
<script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/fingerprint/script/kpf.js?url=/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/fingerprint&token=dcc221cb-d87a-e9e3-5316-7c7a20910bf8"></script>
</body>
</html>

But in the inspection, I got something like this. Maybe I need to add some header information or missing something.

Here is request info and paramerter

If I put this request URL in browser I get the data.

edited Jul 08 '20 at 00:24

asked Jul 08 '20 at 00:15

Shahrear Bin Amin

1,075
13
31

What do get if you run `print(res.status_code)`? – ddastrodd Jul 08 '20 at 00:37
@ddcastrodd I got 429 in status_code – Shahrear Bin Amin Jul 08 '20 at 00:43
That's why you are not getting the data then. 429 is Too Many Requests. – ddastrodd Jul 08 '20 at 01:34
@ddcastrodd How to prevent from too many requests? – Shahrear Bin Amin Jul 08 '20 at 01:35
Take a look at this post: https://stackoverflow.com/questions/22786068/how-to-avoid-http-error-429-too-many-requests-python – ddastrodd Jul 08 '20 at 01:38
@ddcastrodd I got a 429 on my very first request. I then tried various User-Agent headers -- no difference. – Booboo Jul 08 '20 at 01:43
@ddcastrodd I've also tried using header. Doesn't work – Shahrear Bin Amin Jul 08 '20 at 01:56

Not getting data from request in python

0 Answers0