I am trying to parse the html from a website, but some of the data stored in a table is not showing up

Question

I am trying to get the html code from the website: http://www.phila.gov/water/swmap/Parcel.aspx?parcel_id=544393

using the following code:

public class URLGetter {
    private URL url;
    private HttpURLConnection connection;

    public URLGetter(String url) throws MalformedURLException {
        try {
            this.url = new URL(url);

            URLConnection connection2 = this.url.openConnection();
            connection = (HttpURLConnection) connection2;
        } catch (IOException E) {
        }
    }

    public ArrayList<String> getContents() {
        ArrayList<String> contents = new ArrayList<String>();
        try {
            Scanner in = new Scanner(connection.getInputStream());
            while (in.hasNextLine()) {
                contents.add(in.nextLine());
            }
        } catch (Exception e) {
        }
        return contents;
    }

}

Using the very simple testing method:

public class URLTester {

    public static void main(String[] args) throws MalformedURLException {
        URLGetter get = new URLGetter("http://www.phila.gov/water/swmap/Parcel.aspx?parcel_id=544393/print/textversion.html");
        ArrayList<String> list = get.getContents();

        for(String s : list){
            System.out.println(s);
        }
    }

}

I print the html. All of it goes smoothly, except for printing out the data in the tables (inside of the ) brackets. Instead of the various values which should be appearing, for example PERUTO ANGELO CHARLES III, every single value has inside of it &nbsp.

I really don't know why it does this. Looking over the textthat I get by doing this, nothing else is wrong.

edit: I've used this code on other websites and always been able to get the information I need. From this site I get all the information I need, except for the table values.

Answer 1

Do you want to convert HTML codes with equivalent characters? If yes then try below code using StringEscapeUtils#unescapeHtml()

... 
while (in.hasNextLine()) {
    contents.add(org.apache.commons.lang.StringEscapeUtils.unescapeHtml(in.nextLine()));
}
...

Note: require a jar from apache commons

For more info have a look at Replace HTML codes with equivalent characters in Java