scrapy how to get return value from another function

Question

how to return result to the call function? I want my parse2 to return a value back to parse1, so parse1 can store it. right now I am just getting the url value, but not the return value from parse2

def parse1(self, response):
    for lst in response.css("tr.lst-rw"):
        url = lst.css('td[headers="lh_id"] > a::attr(href)').extract_first()
        if url is not None:
            url = response.urljoin(url)
            res = scrapy.Request(url, callback=self.parse2)

        yield{
            'response-date' : res    
        }

def parse2(self, response):
    yield{
        'response date': response.css('div#dnf::text').extract_first().strip()
    }

`parse2` is a generator. I don't think this is correct: https://doc.scrapy.org/en/latest/topics/request-response.html#request-objects — Stephen Rauch, Apr 26 '18 at 23:53

score 0 · Answer 1 · answered Apr 27 '18 at 00:06

0

def parse1(self, response):
      s = self.parse2(response)
      #the rest of your code

def parse2(self, response):
      #your code

answered Apr 27 '18 at 00:06

Abd El Kodous Souissi

357
2
14

scrapy how to get return value from another function

1 Answers1