How get the spider return data with scrapy through an external script?

Question

What is the way to view the return data of the parse function of the spider when I execute a script like this?

from twisted.internet import reactor
from scrapy.crawler import Crawler
from scrapy.settings import Settings
from scrapy import log, signals
from testspiders.spiders.followall import FollowAllSpider

spider = FollowAllSpider(domain='scrapinghub.com')
crawler = Crawler(Settings())
crawler.signals.connect(reactor.stop, signal=signals.spider_closed)
crawler.configure()
crawler.crawl(spider)
crawler.start()
crawler.stats
#log.start()
reactor.run()

I disable the log for view the print messages in the spiders, but with the log enabled the return data don't show either.

The code of the spider parse function return a simple string.

How i get this data? I try to print the "reactor.run" results but always is "none"

Answer 1

This is the way I have found to get the collected items:

items = []
def add_item(item):
    items.append(item)

crawler.signals.connect(add_item, signals.item_passed)

I gave my original answer in the linked question and give a bit more details: https://stackoverflow.com/a/23892650/2730032

Answer 2

If you want to see the logging in the screen change this line:

#log.start()

to this:

log.start(loglevel=log.DEBUG)

to your script.

See this question