I need to write a unit test to the Scrapy spider. The problem is, the only way I know on how to call Scrapy spider programmatically is through scrapy.crawler.CrawlerProcess which creates subprocess, then Twisted reactor, etc. For simple unit test it's a massive overkill.
What I want to do is simply create a request, load project settings somehow, send it and process the response.
Is there a way to do it properly?
EDIT. I checked Scrapy Unit Testing, but the whole point of the test is to check how some xpaths in the database map on the current state of the website. I'm ok with the online testing, I need it actually. (Then it's becoming more like an integration test, but whatever.)
