ScrapySharp has a Web Client able to simulate a real Web browser wrapping the HtmlAgilityPack API.
Questions tagged [scrapysharp]
19 questions
6
votes
1 answer
How to use ScrapySharp to parse elements in an html document?
Here's the project official "Documentation":
https://bitbucket.org/rflechner/scrapysharp/wiki/Home
No matter what I try, I can't find the CssSelect() method that the library is supposed to add to make querying things easier. Here's what I've…
sergserg
- 21,716
- 41
- 129
- 182
5
votes
3 answers
Keep getting stuck loading when ScrapySharp NavigateToPage
My browser just keeps loading when navigatetopage using scrapysharp and won't go to the next line of code. Below is my code using c# asp.net web form. May I know why? The link I use is working and can manually browse. The code just gets stuck at the…
Tiong Gor
- 73
- 1
- 8
3
votes
2 answers
How to find the form in scrapysharp when it only has attributes i.e. no name or id
I am new to scrapySharp as well as web scraping. I am trying to scrape a site that is secured and has a login screen. The form element does not have a name/id attribute, thus making my life more complicated. I have been unable to figure out how…
Dennis Polley
- 33
- 3
2
votes
0 answers
Using scrapy sharp with login
I have created one simple windows app using c# windows forms and scrapysharp library.I have parsed data without problem. But now I need to parse data from another page which requires login.
The problem is I dont know how to save cookies in this…
Hekuran Kastrati
- 21
- 2
2
votes
1 answer
How do I get scrapysharp to work in a MVC web app?
I successfully have scrapysharp working in a console app.
I created a new MVC web app in VS2013 with no authentication or anything else special. I used nuget to add ScrapySharp and then have this code in my Home Controller. I get no response to my…
bhs8227
- 81
- 9
1
vote
0 answers
System.invalidoperationexception sequence contains no elements Web Scraper
I'm trying to create a Web Scraper with ScrapySharp. I encountered the following error -
System.invalidoperationexception sequence contains no elements
Code -
Scraper class:
static void Main(string[] args)
{
…
Leonardo Esquivel
- 57
- 6
1
vote
1 answer
ScrapySharp Form Submit causing System.AggregateException
I spent hours racking my head as to why this isn't working
I'm trying to use ScrapySharp to scrape websites, right now just trying out sample sites then moving to my actual site.
Every time I do a form.Submit() in my program I get hit with a…
arcanium0611
- 123
- 12
1
vote
1 answer
C# ScrapySharp 'System.Net.CookieException: 'The 'Name'='HttpOnly, NID' part of the cookie is invalid.'
So i'm facing an unexpected issue with my code. For some reason, I am unable to download & print the links out of my Google search... Help is much appreciated as I'm really not sure what is going on here... I am also using the DotNET SDK
using…
Natty405454
- 37
- 5
1
vote
1 answer
C# scrape correct web content following jquery
I've been using HtmlAgilityPack for awhile but the web resource I have been working with now has a (seems like) jQuery protocol the browser passes through. What I expect to load is a product page but what actually loads (verified by a WebBrowser…
Xero Phane
- 88
- 8
1
vote
1 answer
Click on HTML elements with Scrapy (WebScraping)
I'm doing a program in c # using scrapySharp or HtmlAgilityPack. But I have the disadvantage of that part of the information that I need, to appear when I click on an HTML element (Button, link ).
In some forums it was commented that when using…
Xime Zabala
- 55
- 7
1
vote
2 answers
ScrapySharp causes Windows Form to freeze without exception
When included in my code
ScrapingBrowser browser = new ScrapingBrowser();
WebPage testPage = browser.NavigateToPage(new Uri("https://www.google.co.uk/"));
causes the Windows Form to stop working (once this line is reached in execution, the form…
KangarooChief
- 381
- 3
- 14