0

I want to schedule a spider from scrapyd-client by giving the command line arguments as well.

e.g: scrapy crawl spider_name -a person="John" -a location="porto" -o local.csv

Above command works well when running spider directly from scrapy, but it does not work when running it from rest API using scrapyd-client.

Basically the question is how to send scrapy's command line arguments like (-a, -o) in scrapyd-client?

Daniyal Faquih
  • 1
  • 2

1 Answers1

0

Use the flag -d instead of the -a flag

msenior_
  • 1,913
  • 2
  • 11
  • 13
  • Hi I tried using -d, but the results were same: `code` **path_to_scrapyd-client schedule -p project_name spider_name -d person=John -d location=porto -o indeed_jobs_data.csv** – Daniyal Faquih Dec 31 '21 at 05:21
  • You should implement the csv saving logic in the spider using `FEEDS` setting in `custom_settings`. Then run command as below `path_to_scrapyd-client schedule -p project_name spider_name -d person=John -d location=porto` – msenior_ Dec 31 '21 at 09:23
  • This command is giving following error: **scrapyd-client: error: unrecognized arguments: -d person=john -d location=porto** – Daniyal Faquih Dec 31 '21 at 11:42