1

Hey I am relatively new to java, and I am trying to make an application that does the following:

  1. Sends a request to a live website
  2. Retrieves the data of that page

For example, assume the following site displays game results, where 'game=500' shows the results for game number 324 of 500 different games. http://www.some-site.com/results.php?game=324

I would like to use a Java program to automatically cycle through the game=1 to game=500, posting to the website and retrieving the results of the page.

What is the best way to do this? Can anyone give me a simple example? If I knew the correct java 'key words', I would google for some tutorials on this concept.

Note: the target-page in question is php

BalusC
  • 1,082,665
  • 372
  • 3,610
  • 3,555
Bob Smith
  • 33
  • 3
  • You can write a `curl` one-liner to do that. That is if it fits your requirements and constraints. – adarshr Jul 29 '11 at 12:25

3 Answers3

3
URL url;
InputStream is = null;
DataInputStream dis;
String line;
for(int i=1;i<=500;i++){
try {
    url = new URL("http://www.some-site.com/results.php?game="+i);
    is = url.openStream();  // throws an IOException
    dis = new DataInputStream(new BufferedInputStream(is));

    while ((line = dis.readLine()) != null) {
        //do sth with the datea
    }
} catch (MalformedURLException mue) {
     mue.printStackTrace();
} catch (IOException ioe) {
     ioe.printStackTrace();
} finally {
    try {
        is.close();
    } catch (IOException ioe) {
        // nothing to see here
    }
 }
}
Sherif elKhatib
  • 45,786
  • 16
  • 89
  • 106
  • this works, how ever the site i am trying to get data from is: http://www.playkeno.com.au/game_results.php if you visit the page, you have to click "i am 18" in order to enter, since it is lottery related. I have to some-how send the 'i am 18' command to the page :( i'm not too sure how to? – Bob Smith Jul 30 '11 at 00:27
  • i used firebug in firefox to check what is being sent. it looks like i need to hit the site: "http://www.playkeno.com.au/index.php?confirmed=nsw&button=I+am+18%2B" – Bob Smith Jul 30 '11 at 01:05
0

Do something like the answer in this other stackoverflow page

and then you want to use a for loop for loop through pages 1 through 500.

Community
  • 1
  • 1
Danny
  • 7,368
  • 8
  • 46
  • 70
0

Apache has some really good Java libraries for accessing HTTP. See this for more details.

Mike Thomsen
  • 36,828
  • 10
  • 60
  • 83