2

I have an idea for an App that would really help me out in work but I'm not sure if it's possible.

I want to run a C# desktop application that will ask for a value. When a value is supplied, the application will open a browswer, go to a webpage and add the value into a form on an online website. The form is then submitted and a new page is loaded that contains a table of results. I then want to extract the table of results from the page source and write code to parse the result values.

It is not important that the user see's this happen in an actual browser. In other words if there's a way to do it by reading HTTP requests then thats great.

The biggest problem I have is getting the values into the form and then retrieving the page source after the form is submitted and the next page loads.

Any help really appreciated.

Thanks

user392810
  • 21
  • 2
  • Do you control the website or are you scraping data off of someone else's site? – Joel Etherton Jul 15 '10 at 14:45
  • The site is a front end for data on one of our servers in work although we aren't allowed direct access for one reason or another. Everything has to go through the web – user392810 Jul 15 '10 at 15:55

5 Answers5

1

This is definitely possible and you don't need to use an actual web browser for this. You can simply use a System.Net.WebClient to send your HTTP request and get an HTTP response.

I suggest to use wireshark (or you can use Firefox + Firebug) it allows you to see HTTP requests and responses. By looking at the HTTP traffic you can see exactly how you should pass your HTTP request and which parameters you should be setting.

Brian R. Bondy
  • 339,232
  • 124
  • 596
  • 636
1

Provided that you're only using this in a legal context:

Usually, web forms are sent via POST request to the web server, specifically some script that handles it. You can look at the HTML code for the form's page and find out the destination for the form (form's action).

You can then use a HttpWebRequest in C# to "pretend you are the form", sending a POST request with all the required parameters (adding them to the HTTP header).

As a result you will get the source code of the destination page as it would be sent to the browser. You can parse this.

Thorsten Dittmar
  • 55,956
  • 8
  • 91
  • 139
0

You don't need to involve the browser with this. WebClient should do all that you require. You'll need to see what's actually being posted when you submit the form with the browser, and then you should be able to make a POST request using the WebClient and retrieve the resulting page as a string.

The docs for the WebClient constructor have a nice example.

spender
  • 117,338
  • 33
  • 229
  • 351
0

See e.g. this question for some pointers on at least the data retrieval side. You're going to know a lot more about the http protocol before you're done with this...

Community
  • 1
  • 1
Pontus Gagge
  • 17,166
  • 1
  • 38
  • 51
0

Why would you do this through web pages if you don't even want the user to do anything? Web pages are purely for interaction with users, if you simply want data transfer, use WCF.

@Brian using Wireshark will result in a very angry network manager, make sure you are actually allowed to use it.

MrFox
  • 4,852
  • 7
  • 45
  • 81