Request

Asked Aug 07 '20 at 10:25

Active Aug 07 '20 at 15:53

Viewed 392 times

I was making a request to the following API URL but getting a <403> response.

The link works fine when opening with the browser. I opened the scrapy response in the browser and saw that it's showing a captcha. How can I bypass the captcha and make a request like a real human with scrapy or request?

# -*- coding: utf-8 -*-
import scrapy
import json

class MainSpider(scrapy.Spider):
    name = 'main'
    start_urls = ['https://www.mediamarkt.de/api/v1/graphql?operationName=CategoryV3&variables={%22hasMarketplace%22%3Atrue%2C%22wcsId%22%3A%22459521%22%2C%22page%22%3A1%2C%22experiment%22%3A%22mp%22}&extensions={%22pwa%22%3A{%22salesLine%22%3A%22Media%22%2C%22country%22%3A%22DE%22%2C%22language%22%3A%22de%22}%2C%22persistedQuery%22%3A{%22version%22%3A1%2C%22sha256Hash%22%3A%2211d83e8b68f9a2a73adc1cf50dd17b1dffcc99dc9636285bfc42088860610b05%22}}']

    def parse(self, response):
        resp = json.loads(response.body)
        values = resp.get('data').get('products')

        print(values)

edited Aug 07 '20 at 15:53

asked Aug 07 '20 at 10:25

codewithawais

1

Can you provide the actual link to the API? You've just provided the link to an image. – Mooncrater Aug 07 '20 at 14:25
I just updated the question with the link. – codewithawais Aug 07 '20 at 15:54
Have you taken a look at this question? https://stackoverflow.com/questions/39137559/how-do-i-set-up-scrapy-to-deal-with-a-captcha – Ryan Aug 07 '20 at 17:06
Just checked but it didn't work for me. – codewithawais Aug 07 '20 at 17:21

<403> Response by an API URL Python Scrapy/Request

0 Answers0