How to add Django_app.models inside items.py of Scrapy my project

Question

my file structure, both django folder(mysite) and scrapy folder(intex) are in same directory.

| mysite (django folder)
  | mysite
     | settings.py
     | urls.py
  | polls
     | admin.py
     | models.py
  | db.sqlite3
  | manage.py

| intex (scrapy folder)
  | intex
     | spider
        | spiders.py
     | items.py

here's my models.py file:

from django.db import models

class ServiceCenters(models.Model):

    brand_name = models.CharField(max_length=50,blank=True,default='')

    service_centre_city = models.CharField(max_length=50,blank=True,default='')

here's my items.py file:

import scrapy
from scrapy_djangoitem import DjangoItem
from scrapy.item import Item, Field
from polls.models import ServiceCenters

class IntexItem(DjangoItem):
    django_model = ServiceCenters

I want to save my data into django server, While I'm running this code scrapy crawl scrawl_name I'm getting this error:

from polls.models import ServiceCenters
ImportError: No module named polls.models

Does this answer your question? [Scrapy and Django import error](https://stackoverflow.com/questions/19164482/scrapy-and-django-import-error) — Gallaecio, Nov 25 '19 at 10:04

score 3 · Answer 1 · answered Dec 08 '15 at 19:33

You need to tell python where to find the polls.models module. First you need to add the folder containing your django app to the PYTHONPATH and second, configure django to be able to access your models.

This is the code snippet to achieve this:

import os
import sys
import django

sys.path.append('...absolute path to mysite...')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')
django.setup()

This code needs to run before you import your models, i mean, before the line:

from polls.models import ServiceCenters

Hope this helps you

How to add Django_app.models inside items.py of Scrapy my project

1 Answers1