1

I am trying to make a simple Android app that can retrieve lottery numbers from a website (https://www.lottostat.dk/rssfeed.php). I have tried using the example code provided here (and inserted below): Using Java to pull data from a webpage?

The example code works great when using the original target website (Using Java to pull data from a webpage?) and I can read the entire underlying html code in the output in Android Studio. But when I change the target website to the one I want to get my data from (https://www.lottostat.dk/rssfeed.php) there is no output (br.readLine() returns null).

What could be the problem here? Do I perhaps need a different solution for reading the .php website (even though the underlying code seems to be plain XML)?

Here's the working original sample code for reference:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;


public class DownloadPage {

    public static void main(String[] args) throws IOException {

        // Make a URL to the web page
        URL url = new URL("http://stackoverflow.com/questions/6159118/using-java-to-pull-data-from-a-webpage");

        // Get the input stream through URL Connection
        URLConnection con = url.openConnection();
        InputStream is =con.getInputStream();

        // Once you have the Input Stream, it's just plain old Java IO stuff.

        // For this case, since you are interested in getting plain-text web page
        // I'll use a reader and output the text content to System.out.

        // For binary content, it's better to directly read the bytes from stream and write
        // to the target file.


        BufferedReader br = new BufferedReader(new InputStreamReader(is));

        String line = null;

        // read each line and write to System.out
        while ((line = br.readLine()) != null) {
            System.out.println(line);
        }
    }
}
daniel
  • 23
  • 8

2 Answers2

1

Apparently, this website is user-agent dependent. Adding User-Agent header solves the problem. Try to use

    URLConnection con = url.openConnection();
    con.setRequestProperty("User-Agent", "Mozilla/5.0");
    InputStream is =con.getInputStream();
0

Add a user-agent, that should do the trick (tested with android 5.1.1 device):

URL url = new URL("https://www.lottostat.dk/rssfeed.php");
URLConnection con = url.openConnection();
con.setRequestProperty("User-Agent", "Mozilla");

Alternative: use jsoup

Document doc = Jsoup.connect("https://www.lottostat.dk/rssfeed.php").userAgent("Mozilla").get();          
String content = doc.toString();
Frederic Klein
  • 2,846
  • 3
  • 21
  • 37
  • Thanks! It works when I specify the user-agent agent.How does one know if setting the user-agent to e.g. Mozilla is necessary? – daniel Aug 12 '16 at 18:19
  • @daniel I suppose no way if you don't have access to the server. I would recommend you to use software like Fiddler in such cases: when in browser data is displayed as it should but in app there's nothing; Then you capture request to server that returns data, compare to one issued by app and find differences. – Nikita Khlebushkin Aug 12 '16 at 18:27
  • But it usually doesn't hurt to specify a user agent. Or you check if the content returns empty and then retry with a user agent. – Frederic Klein Aug 12 '16 at 20:10