Simulate click on html?

Question

I'm using HtmlAgilityPack for get some soccer events from this site.

The event that I'm grabbing are inside the All tab. So essentially what I did is get the table where all events are located like this:

string url = "http://it.soccerway.com/";
string data = new WebClient().DownloadString(url);
var doc = new HtmlDocument();
doc.LoadHtml(data);
var table = doc.DocumentNode.SelectSingleNode("//table[@class='matches date_matches grouped ']");

in the next time I get all the visible event, so all the div that have the class group-head expanded loaded:

var tableTrHeader = table.SelectNodes("//tr[@class='group-head expanded loaded  ']");

and next iterate it. All of this working good but I've a problem. Infact there are other event in the table, but unfortunately this doesn't have the class loaded but simply have: group-head clickable.

So I guess there is something in the js code of the site that perform an action or something like to get the details of clicked row.

I thought to load an html that have all the item expanded, but unfortunately I doesn't know an approach that allow me to send a click action on a specific target html element through c#. I suppose that HtmlAgilityPack is not done for this target but only for scraping.

Someone have a workaround for this? Thanks.

you can perform click event and other action using WebBrowser in windows form — Lalit Rajput, Sep 02 '16 at 11:20
Okay, Actually i did it in Windows Form App by using WebBrowser class. — Lalit Rajput, Sep 02 '16 at 11:23
WPF allows hosting of WebBrowser or a similar control as well. — kenny, Sep 02 '16 at 11:29
@kenny I know, in the past I've used WebBrowser with vb.net but I remember that this is so slow to scrape data. — AgainMe, Sep 02 '16 at 11:31

score 2 · Accepted Answer · answered Sep 02 '16 at 11:25

2

I suppose that HtmlAgilityPack is not done for this target but only for scraping.

Right.

Someone have a workaround for this?

It heavily depends on how it is implemented. If it's JavaScript, then good luck. You probably need to switch your whole tool chain and use Browser automation instead.

If it's a HTML clickable link, get the link, make another request and parse it with HtmlAgilityPack again.

answered Sep 02 '16 at 11:25

Thomas Weller

55,411
20
125
222

Looking the site structure is all based on json, when I click on the row an http request is made and new details is loaded. So is all based on js. Unfortunately. – AgainMe Sep 02 '16 at 11:28

score -2 · Answer 2 · edited May 23 '17 at 12:24

I found this:

using System;
using System.Windows.Forms;
using System.Runtime.InteropServices;

public class Form1 : Form
{
     [DllImport("user32.dll",CharSet=CharSet.Auto, CallingConvention=CallingConvention.StdCall)]
     public static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint cButtons, uint dwExtraInfo);

     private const int MOUSEEVENTF_LEFTDOWN = 0x02;
     private const int MOUSEEVENTF_LEFTUP = 0x04;
     private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
     private const int MOUSEEVENTF_RIGHTUP = 0x10;

public Form1()
{
}

public void DoMouseClick()
{
  //Call the imported function with the cursor's current position
  int X = Cursor.Position.X;
  int Y = Cursor.Position.Y;
  mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, X, Y, 0, 0);
}

//...other code needed for the application
}

Here, take a look :)

Nope, this simulate a click on the window, I'm looking for something that simulate the click when the http request is sent, so I can get the html with all the classes `expanded loaded` — AgainMe, Sep 02 '16 at 11:25

Simulate click on html?

2 Answers2