Parse SSL encrypted website on Android

Question

There is a SSL encrypted (signed by thawte, so no selfsigned problems) Website with login required. My goal is to read some data from a subpage of this site. The regular process on a webbrowser would be:

go to "https://abc.de"
enter credentials and hit button
now .. logged in you click on a link and can see the data

Now in Android, I want the user to click a button, do all that stuff in the background automatically, and show him results in a parsed form.

As I dont know much about Android programming, my first guess was the WebView. When sending a POST request directly to the login page with the credentials, this works fine. But it seems like I cant get the sourcecode of the WebView. What would be the best/easiest technology/way to do what I want?

score 2 · Accepted Answer · answered Oct 18 '12 at 11:06

Rather than using just the WebView, it's probably better to use either HttpClient or HttpURLConnection (see this blog post for a discussion of which to use) to initially load the page from the server. You can then do what you need to do with the data returned and then pass it to a WebView for display.

score 2 · Answer 2 · edited May 23 '17 at 12:14

2

You can use HttpClient or HttpsUrlConnection to communicate directly with the server without the need for a WebView. Have a look at my question/answer for an example.

edited May 23 '17 at 12:14

Community

1
1

answered Oct 18 '12 at 12:05

dave.c

10,910
5
39
62

score 0 · Answer 3 · answered Oct 18 '12 at 11:17

Can you submit the form for this user ? You can use a HTTPClient and open a connection to submit the form. However this can easily get tricky and ugly. Some servers pass a nonce to a browser before allowing a client to login. If your server has such security features, you will end up writing quite a lot of code before getting the programmatic login to succeed (if at all). Furthermore you need to use a HTML parser like [1] to even read the data.

Its much easier to ask the website to expose APIs to retrieve this data, if that is possible. This would be the route I'd suggest. A web service can also authenticate your user through OAuth or BASIC auth over HTTPS. It takes away the pain of screen scraping but also requires some investment in time to write said service.

[1] - http://htmlcleaner.sourceforge.net/

Parse SSL encrypted website on Android

3 Answers3