.net framework with scrapy python

Question

Is it possible to use .NET framework with the Python scrapy framework to scrape data from different sites? I am working on my final year project in which I want to use C# as front end language and Python for scraping the data.

Maybe this helps http://stackoverflow.com/a/3964225/2382792 – ρss May 07 '14 at 09:08 — ρss, May 07 '14 at 09:08
I am interested in this too. Can I use scrapy from .net? – jimbo May 21 '14 at 17:17 — jimbo, May 21 '14 at 17:17

score 3 · Answer 1 · edited May 23 '17 at 11:53

3

I don't think it is possible, because Scrapy uses twisted networking engine which can not run on IronPython

Alternatively, you may start your spider/crawler through command line using C# and then interact with it using JSON API

edited May 23 '17 at 11:53

Community

1
1

answered Aug 21 '14 at 17:53

Adil

49
1
3

Rory · Answer 2 · 2017-12-18T22:43:02.783

2

Take a look at ScrapySharp, also described on this blog, which is the C# version of python's Scrapy.

edited Dec 18 '17 at 22:43

answered Dec 18 '17 at 10:30

Rory

40,559
52
175
261

.net framework with scrapy python

2 Answers2

Linked