Android: Reading the HTML of a webpage into a String

Question

I am new to Android Development and I am trying to read the HTML of a webpage and store it in a String ("myHTML") below, and display it on the app.

However the application ends whenever it is run. I have been trawling the internet for the reason for this and have come across some articles saying that internet access cannot be done in the main UI thread of the app due to its "expensive" nature. Has anyone come across a similar problem before? I would be grateful for any further information on this problem... at a beginners level :)

Here is the program:

import android.app.Activity;
import android.os.Bundle;
import android.os.Handler;
import android.widget.TextView;
import java.util.ArrayList;
import java.util.regex.*;
import java.net.*;
import java.io.*;

/*
   * Gets A webpage's HTML and saves to a string
   */
public String WebPageToHTML(String Webpage) throws   IOException{
  URL x = new URL(Webpage);
    BufferedReader in = new BufferedReader(
          new InputStreamReader(
          x.openStream()));
    String y = "";
    String inputLine;
    while ((inputLine = in.readLine()) != null)
       y = y.concat(inputLine);
     in.close();
  return y;     
}

public void onCreate(Bundle savedInstanceState) {
      super.onCreate(savedInstanceState);
      setContentView(R.layout.main);
      TextView tv = new TextView(this);

       String FirstAddress = "http://www.google.com";
       String myHTML = "";
  try {
    myHTML = WebPageToHTML(FirstAddress);
  } catch (IOException e) {
    e.printStackTrace();
  }    
      tv.setText(myHTML);
      setContentView(tv);
}

LOGCAT:

12-29 14:41:44.441: E/AndroidRuntime(540): java.lang.RuntimeException: Unable to start activity ComponentInfo{my.first.app/my.first.app.WhatHaveIMissedActivity}: android.os.NetworkOnMainThreadException
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1956)
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1981)
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.app.ActivityThread.access$600(ActivityThread.java:123)
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1147)
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.os.Handler.dispatchMessage(Handler.java:99)
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.os.Looper.loop(Looper.java:137)
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.app.ActivityThread.main(ActivityThread.java:4424)
12-29 14:41:44.441: E/AndroidRuntime(540):  at java.lang.reflect.Method.invokeNative(Native Method)
12-29 14:41:44.441: E/AndroidRuntime(540):  at java.lang.reflect.Method.invoke(Method.java:511)
12-29 14:41:44.441: E/AndroidRuntime(540):  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784)
12-29 14:41:44.441: E/AndroidRuntime(540):  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551)
12-29 14:41:44.441: E/AndroidRuntime(540):  at dalvik.system.NativeStart.main(Native Method)
12-29 14:41:44.441: E/AndroidRuntime(540): Caused by: android.os.NetworkOnMainThreadException
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.os.StrictMode$AndroidBlockGuardPolicy.onNetwork(StrictMode.java:1099)
12-29 14:41:44.441: E/AndroidRuntime(540):  at java.net.InetAddress.lookupHostByName(InetAddress.java:391)
12-29 14:41:44.441: E/AndroidRuntime(540):  at java.net.InetAddress.getAllByNameImpl(InetAddress.java:242)
12-29 14:41:44.441: E/AndroidRuntime(540):  at java.net.InetAddress.getAllByName(InetAddress.java:220)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpConnection.<init>(HttpConnection.java:71)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpConnection.<init>(HttpConnection.java:50)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpConnection$Address.connect(HttpConnection.java:351)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpConnectionPool.get(HttpConnectionPool.java:86)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpConnection.connect(HttpConnection.java:128)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpEngine.openSocketConnection(HttpEngine.java:308)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpEngine.connect(HttpEngine.java:303)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpEngine.sendSocketRequest(HttpEngine.java:282)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpEngine.sendRequest(HttpEngine.java:232)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:273)
12-29 14:41:44.441: E/AndroidRuntime(540):  at libcore.net.http.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:168)
12-29 14:41:44.441: E/AndroidRuntime(540):  at java.net.URL.openStream(URL.java:462)
12-29 14:41:44.441: E/AndroidRuntime(540):  at my.first.app.WhatHaveIMissedActivity.WebPageToHTML(WhatHaveIMissedActivity.java:71)
12-29 14:41:44.441: E/AndroidRuntime(540):  at my.first.app.WhatHaveIMissedActivity.onCreate(WhatHaveIMissedActivity.java:99)
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.app.Activity.performCreate(Activity.java:4465)
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1049)
12-29 14:41:44.441: E/AndroidRuntime(540):  at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1920)
12-29 14:41:44.441: E/AndroidRuntime(540):  ... 11 more

score 12 · Answer 1 · edited May 23 '17 at 12:27

12

You can use the HttpClient to request this information. It will be done synchronously, however you can make asynchronous requests as well.

String myUri = "http://www.whatever.com";
HttpClient httpClient = new DefaultHttpClient();
HttpGet get = new HttpGet(myUri);

HttpResponse response = httpClient.execute(get);

// Build up result
String bodyHtml = EntityUtils.toString(response.getEntity());

You will also need to add the following to your app's manifest file.

<uses-permission android:name="android.permission.INTERNET" />

A good thread on how to wrap this with an AsyncTask is: Common class for AsyncTask in Android?

edited May 23 '17 at 12:27

Community

1
1

answered Dec 29 '11 at 15:51

Khaos

131
6

Hi, thanks for your answer. I have tried using the code above to perform this synchronously but the problem remains. I will have a look at "Common class for AsyncTask in Android?" now. – B. Bowles Dec 29 '11 at 16:16
You're welcome. Because the call by default will block your thread, onCreate is most likely resulting in an Activity Not Responding (ANR). As you have indicated, the solution is to have it run asynchronously, and in the completed callback, perform whatever UI update actions are needed! – Khaos Dec 29 '11 at 17:07
which library do I need to import to get `HttpClient`? (Android Studio doesn't know) – falsePockets Jun 16 '18 at 07:59

score 1 · Answer 2 · answered Dec 29 '11 at 15:47

1

One first issue is that you are blocking the UI thread when you request the page. Chances are you receive an ANR during the onCreate. Try using an AsyncTask for that sort of things.

Also, make sure you have declared the INTERNET permission in the Manifest.

Please post your logcat output so we can have a better idea of the problem.

answered Dec 29 '11 at 15:47

njzk2

38,969
7
69
107

I have declared the INTERNET permission in the Manifest and the logcat output is now available. I will have a look into AsyncTasks now. Thanks for your help! – B. Bowles Dec 29 '11 at 15:51
1

according to your logcat, android.os.NetworkOnMainThreadException is exactly that: the networking is done on the main thread, which is forbidden. – njzk2 Jan 02 '12 at 15:04

Android: Reading the HTML of a webpage into a String

2 Answers2