0

I'm trying to run a scrapy spyder but I can't import my pipeline Item

Library used

import scrapy
import json
from functools import reduce
from scrapy.selector import Selector
from anime_reviews.items import AnimereviewsItem
last_page = 1789

The Spyder class is (I didn't put the full code of class):

class AnimeReviewsSpider(scrapy.Spider):
name = 'AnimeReviews_spider'
allowed_urls =['myanimelist.net']
start_urls = ['https://myanimelist.net/reviews.php?t=anime']

def parse(self, response):
    page_urls = [response.url + "&p=" + str(pageNumber) for pageNumber in range(1, last_page+1)]
    #page_urls = ['https://myanimelist.net/reviews.php?t=anime']
    for page_url in page_urls:
        yield scrapy.Request(page_url,
            callback = self.parse_reviews_page)

def parse_reviews_page(self, response):
    item = AnimereviewsItem()
    reviews = response.xpath('//*[@class="borderDark pt4 pb8 pl4 pr4 mb8"]').extract()       #each page displays 50 reviews

    for review in reviews:
        anime_title = Selector(text = review).xpath('//div[1]/a[1]/strong/text()').extract()
        anime_url = Selector(text = 

.....
    yield item

And Items class is :

class AnimereviewsItem(scrapy.Item):
    anime_title = scrapy.Field()
    review_time = scrapy.Field()
    reviewer = scrapy.Field()
    rating = scrapy.Field()
    review_text = scrapy.Field()
    pic_url = scrapy.Field()
    anime_url = scrapy.Field()

Then when i'm trying to AnimeReviewsItem i had " ModuleNotFoundError: No module named 'anime_reviews'

My project path is :

anime_reviews/scrapy.cfg

  • anime_reviews/scrapy.cfg/anime_reviews/...
  • .../ init.py , items.py , middlewares.py , pipelines.py , settings.py , pycache , spyders/...
  • .../ pycache , init.py , anime_reviews_spider.py

error :

   runfile('C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders/anime_reviews_spider.py', wdir='C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders')
Traceback (most recent call last):

  File "<ipython-input-13-53f793bce8a2>", line 1, in <module>
    runfile('C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders/anime_reviews_spider.py', wdir='C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders')

  File "C:\Users\alexa\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "C:\Users\alexa\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/alexa/Desktop/Projet recommandation/anime_reviews/anime_reviews/spiders/anime_reviews_spider.py", line 13, in <module>
    from anime_reviews.items import AnimereviewsItem

ModuleNotFoundError: No module named 'anime_reviews'
user9176398
  • 441
  • 1
  • 4
  • 15

0 Answers0