2

I've been trying to use scrapy-crawlera as a proxy for scraping some data with scrapy. I've added these rows in settings.py:

DOWNLOADER_MIDDLEWARES = { 'scrapy_crawlera.CrawleraMiddleware': 610, } 
CRAWLERA_ENABLED = True 
CRAWLERA_APIKEY = 'MY_CRAWLERA_API_KEY'

In the newly changed scrapinghub website app.zyte.com i have created a Smart proxy manager account and my api key is from my account settings.

I get this warning WARNING: Retrying crawlera request for authentication issue and the websites doesn't get scraped.

Is there something I'm doing wrong?

memeister
  • 53
  • 5
  • Well you're following the docs exactly so seems like either you have an invalid api key or there's an issue with Zyte. I would contact Zyte support. – Felix Eklöf Mar 09 '21 at 09:23

1 Answers1

1

This may seem like an obvious answer but I had this issue and solved it by making sure the API key I used was correct. In my case, the first API key Zyte issued to me wasn't activated. I went back to my account console and copied the API key there (it was different).

So, ensure you're using the right key or try creating a new one in the console.

aboutaaron
  • 4,869
  • 3
  • 36
  • 30