Android programming and using HTML parsers to pull category data from a Wikia.com page for use in a ListView

Question

first off I was wondering if there are even HTML parsers that work with Android app programming or not, or if all I have access to is the html commands listed on the Android developers web site. The reason is I am making an app that allows you to access the Zelda Wikia and instead of hard coding everything such as the titles of the video games I wanted to go ahead and pull the names using the MediaWiki API, the command I found was this:

http://zelda.wikia.com/api.php?action=query&list=categorymembers&cmtitle=Category:Games&cmlimit=500

This returns all of the titles in HTML formatting and the only way that I can think to pull what I need from what this returns is using an HTML parser, but I am not sure that there is one that works with Android programming and if so how I would even go about pulling what I need from this. After I get the data from this I want to display it in a ListView and when the title is clicked it will take the user to the Wikia for that specific title in a WebView. Is this the way I should be going about this or are there any other recommendations that anyone has, please I really need help. The rest of the code is as follows just incase anyone wants to see what I have:

    package com.lvlup.kikurself.zeldatest;

    import android.app.ListActivity;
    import android.content.Intent;
    import android.os.Bundle;
    import android.view.View;
    import android.widget.AdapterView;
    import android.widget.ArrayAdapter;
    import android.widget.ListView;
    import android.widget.AdapterView.OnItemClickListener;

    public class zeldaGames extends ListActivity {
        public void onCreate(Bundle icicle) {
        super.onCreate(icicle);
        String[] values = new String[] { "The Legend of Zelda", "Zelda II: The
         Adventure of Link", "The Legend of Zelda: Ocarina of Time",};
        ArrayAdapter<String> adapter = new ArrayAdapter<String>(this,
        android.R.layout.simple_list_item_1, values);
        setListAdapter(adapter);
        final ListView zeldaList = getListView();

        zeldaList.setOnItemClickListener(new OnItemClickListener() {
        public void onItemClick(AdapterView<?> parent, View v, int position, long thisID)
      {
         Object o = (zeldaList.getItemAtPosition(position));
         String gameName_temp = (o.toString());

         Intent newIntent = new Intent(v.getContext(), gameDisp.class);
         newIntent.putExtra("tempG", gameName_temp);
         startActivity(newIntent);

    }
    });

Thank you for taking the time to read this.

Check here http://stackoverflow.com/questions/2188049/parse-html-in-android . There are plenty of html parsers and usage examples around the web. Searching "android html parser" found me tons of results including the link i posted for you — dymmeh, May 10 '12 at 20:13

score 2 · Answer 1 · answered May 10 '12 at 20:03

2

I wouldn't use a parser on the phone. Before you know, the owner of the page could come up with a new layout, ruining your app.

Instead, I would build a webservice (PHP with DomXML is an exelent choice) which parses the given site, and returns file in XML, JSON or other format. And then I would write an app to use the parser webservice as a datasource.

In this way, you will have a lot more computer power. You only have to maintain one instance of the parser, an you know it works on any device, that has downloaded your app.

Maybe sounds like a lot of work, but trust me on this - you'll be better of. Personal and profesional experience

answered May 10 '12 at 20:03

hogni89

1,920
6
22
39

What you're suggesting implies that your web service magically wouldn't be affected by a html redesigned by the site owner. Additionally, this requires them to host a web server for their application to get results from. That costs money(generally.. i'm sure their might be free ones) and most android developers are developing for fun (aka. not gonna buy a server). And unless the html is massive.. as in dozens of pages.. the phone has plenty of power to parse it. Personal and professional experience ;) – dymmeh May 10 '12 at 20:11
However, I'm not saying you're wrong here. What you said would work great if someone has the resources/knowledge to do so. I just feel like you're going overkill on it when a HTML parser would be fine. – dymmeh May 10 '12 at 20:23
@dymmeh: "... You only have to maintain one instance of the parser ...". And another plus is, that if kikurself wants a second datasource, it's easy enough to add. And a hosted PHP server costs as little as $1/month – hogni89 May 10 '12 at 20:26
1

I understood your point with the parser. I fully agree with it too. My point was that most people won't want to deploy servers when they can simply use a HTML parser and be done with it. If there are cheap enough / free web servers that can accept a high amount of web requests .. sure why not. – dymmeh May 10 '12 at 20:34

score 1 · Accepted Answer · answered May 11 '12 at 08:52

1

Append &format=xml to your URL to get machine-readable data. See MW help on API formats at https://www.mediawiki.org/wiki/API:Data_formats and general API help at https://www.mediawiki.org/wiki/API

answered May 11 '12 at 08:52

MaxSem

3,457
21
34

Hello I was not aware of this thank you. Just a quick question though, if I do this and get this returned in XML format, is XML easier to work with than HTML to parse to get the data I need out of it? – kikurself May 11 '12 at 19:04
@kikurself XML is much easier to work with because the data is structured in a way that is meant to be easily read and easily parsed by code. HTML on the other hand is meant to be parsed by a browser and doesn't really care if a developer is trying to pull info from it. Using XML will be significantly easier and give you less change of error – dymmeh May 14 '12 at 19:46

Android programming and using HTML parsers to pull category data from a Wikia.com page for use in a ListView

2 Answers2