2

I am using Scrapy inside a Django project.

Inside my pipelines.py file, I can import a model from Django without any problem:

from competitions.models import Competition

I can display the list of the attributes of the Competition model successfully:

for field_name in Competition._meta.get_all_field_names():
    print field_name #>>> OK

However, all the methods of Competition.objects such as Competition.objects.get() raise django.db.utils.DatabaseError: no such table: competitions_competition"

For instance, I get this error when I try to get an instance:

uefa_champ_leagues = Competition.objects.get(code='EUR_C1_2013')

I have applied the recommendations from Saving Django model from Scrapy project and Access django models inside of Scrapy and since the import works, I doubt that the problem comes from there.

Any idea ?

Community
  • 1
  • 1
Mathieu
  • 431
  • 1
  • 6
  • 15
  • did you run `syncdb`? do the tables exist in the database? – sha256 Dec 14 '13 at 11:48
  • "yes" and "yes" the whole Django project has been running fine for months. I am just testing Scrapy instead of BeautifulSoup. – Mathieu Dec 14 '13 at 11:50
  • 1
    How about using Scrapy from within a custom django admin command? https://docs.djangoproject.com/en/dev/howto/custom-management-commands/ – janos Dec 14 '13 at 12:31
  • @janos: thanks for the reference, I'll give it a try – Mathieu Dec 14 '13 at 12:52
  • 1
    @janos: not only is django admin command a great discovery thanks to you, but it also solved my problem. Thank you very much ! – Mathieu Dec 15 '13 at 01:13
  • Thanks @Mathieu, I turned it into a proper answer, as this is a common mistake to make (I did it too) – janos Dec 15 '13 at 05:23

1 Answers1

1

It is a common mistake to try to hack together a similar environment like python manage.py shell gives you. It's dirty and confusing and leads to problems like in the OP: it seems that Django cannot find the correct location of the database file, or something like that. (And you're not alone, I've made this mistake too in the beginning.)

The recommended solution for writing scripts that work with your Django site is to create a custom django-admin command, so that you can run it using python manage.py yourcommand. It's clean, testable, and the logic belongs to your Django project anyway.

janos
  • 120,954
  • 29
  • 226
  • 236