1

I'm developing a simple web application in Django, and I wanted to start a Scrapy spider from an admin action.

Athari
  • 171
  • 3
  • 11

1 Answers1

2

I'd start with integrating a normal task management process with your django project, see celery package:

Celery is a simple, flexible and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system.

Also, see how to use celery with django.

Then, I'd create a celery task where Scrapy is invoked, see:

Also see:

Hope that helps.

Community
  • 1
  • 1
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
  • I installed (celery , django_celery and RabbitMQ) . and my first ts=est with celery didn't work (http://celery.readthedocs.org/en/latest/getting-started/first-steps-with-celery.html) . when I Call the task in python shell it return "no module named tasks" . – Athari Mar 14 '14 at 17:15
  • @Athari it's a separate problem - better start a new thread and provide error details. Thanks. – alecxe Mar 14 '14 at 17:17