How can I get html source in Android?

Question

I need to extract web page source to string in Android. I tried to do it using HttpClient, HttpGet and HttpResponse, but this method doesn't work. I have to do try/catch to every initialization of this methods and anyway, the application force closes.

  public static String getHtmlSource(String webPage){
            HttpClient client = new DefaultHttpClient();
            URI url = null;
            try {
                url = new URI("http://www.kinopoisk.ru/film/581493");
            } catch (URISyntaxException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            HttpGet request = new HttpGet(url);
            HttpResponse response = null;
            try {
                response = client.execute(request);
            } catch (ClientProtocolException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }

            String html = "";
            InputStream in = null;
            try {
                in = response.getEntity().getContent();
            } catch (IllegalStateException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            BufferedReader reader = new BufferedReader(new InputStreamReader(in));
            StringBuilder str = new StringBuilder();
            String line = null;
            try {
                while((line = reader.readLine()) != null)
                {
                    str.append(line);
                }
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            try {
                in.close();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            html = str.toString();
            return html;
        }

LogCat

09-18 19:28:07.989: W/dalvikvm(5708): threadid=1: thread exiting with uncaught exception (group=0x41f64300)
09-18 19:28:08.029: E/AndroidRuntime(5708): FATAL EXCEPTION: main
09-18 19:28:08.029: E/AndroidRuntime(5708): java.lang.RuntimeException: Unable to start activity ComponentInfo{com.gavrilov.egor.movies/com.gavrilov.egor.movies.MoviesList}: android.os.NetworkOnMainThreadException
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2059)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2084)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.app.ActivityThread.access$600(ActivityThread.java:130)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1195)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.os.Handler.dispatchMessage(Handler.java:99)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.os.Looper.loop(Looper.java:137)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.app.ActivityThread.main(ActivityThread.java:4745)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at java.lang.reflect.Method.invokeNative(Native Method)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at java.lang.reflect.Method.invoke(Method.java:511)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at dalvik.system.NativeStart.main(Native Method)
09-18 19:28:08.029: E/AndroidRuntime(5708): Caused by: android.os.NetworkOnMainThreadException
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.os.StrictMode$AndroidBlockGuardPolicy.onNetwork(StrictMode.java:1117)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at java.net.InetAddress.lookupHostByName(InetAddress.java:385)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at java.net.InetAddress.getAllByNameImpl(InetAddress.java:236)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at java.net.InetAddress.getAllByName(InetAddress.java:214)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:137)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:164)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:119)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:360)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:555)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:487)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:465)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at com.gavrilov.egor.movies.ParseHtmlPage.getHtmlSource(ParseHtmlPage.java:32)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at com.gavrilov.egor.movies.MoviesList.onCreate(MoviesList.java:30)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.app.Activity.performCreate(Activity.java:5008)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1079)
09-18 19:28:08.029: E/AndroidRuntime(5708):     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2023)

It will be a lot easier to try and help you if you post the actual *code* that is giving you errors. — kaderud, Sep 18 '12 at 15:47
You expect someone telling you what you are doing wrong without showing us what you are doing (in terms of code)? — Darin Dimitrov, Sep 18 '12 at 15:48

Alexander Kulyakhtin · Accepted Answer · 2012-09-18T15:55:55.517

1

Can it be you don't have INTERNET permission in the manifest?

Edit: Ah, you are doing that on the main thread, no?

edited Sep 18 '12 at 15:55

answered Sep 18 '12 at 15:50

Alexander Kulyakhtin

47,782
38
107
158

Remove those try/catch and show your LogCat output. It will probably point out the exception line pretty clear. And maybe you're doing that on the main thread? – Alexander Kulyakhtin Sep 18 '12 at 15:55
Without try/catches this doesn't work too – Egor Gavrilov Sep 18 '12 at 15:57
Of course it doesn't but what does LogCat print? And - you shouldn't do it on the main thread (in case you do) – Alexander Kulyakhtin Sep 18 '12 at 15:58
Yes, I do it in the main thread. I'll try to solve this problem. – Egor Gavrilov Sep 18 '12 at 16:01

score 0 · Answer 2 · edited May 23 '17 at 11:49

0

See this issue to solve exception: How to fix android.os.NetworkOnMainThreadException?. For HTML parsing you can use: http://jsoup.org/

edited May 23 '17 at 11:49

Community

1
1

answered Sep 18 '12 at 16:08

Anis BEN NSIR

2,555
20
29

How can I get html source in Android?

2 Answers2