HttpURLConnection reading response content on 403 error

Question

When I fetch data from an URL with a 403 response

is = conn.getInputStream();

It throws an IOException and I can't get the response data.

But when I use firefox and access that url directly, The ResponseCode is still 403, but I can get the html content

score 78 · Accepted Answer · answered Jan 08 '11 at 08:14

78

The HttpURLConnection.getErrorStream method will return an InputStream which can be used to retrieve data from error conditions (such as a 404), according to the javadocs.

answered Jan 08 '11 at 08:14

coobird

159,216
35
211
226

4

No, it won't, for the code of the function contains only 'return null;' line. (Java 6,7) – Gangnus Mar 25 '14 at 13:56
2

@Gangnus Read the Javadoc carefully: "If the connection was not connected, or if the server did not have an error while connecting or if the server had an error but no error data was sent, this method will return null. This is the default." Otherwise (errors 4xx), you will get the stream to read from. – Miljen Mikic Jun 05 '14 at 09:20
@MiljenMikic The difference between code and Javadoc means only that the last one is erroneous. – Gangnus Jun 05 '14 at 11:43
4

@Gangnus HttpURLConnection is *abstract* class. Concrete implementation works exactly as explained in Javadoc. – Miljen Mikic Jun 06 '14 at 07:48
@MiljenMikic It is abstract. But this function already IS there as I say. There is no "missing implement exception". And concrete implementation works as you write it. You CAN override the parent's function, but you needn't. The Javadoc is erroneous and helps to make more errors. – Gangnus Jun 06 '14 at 08:26

qwertzguy · Answer 2 · 2014-11-11T23:07:32.403

Usage example of HttpURLConnection :

String response = null;
try {
    URL url = new URL("http://google.com/pagedoesnotexist");
    HttpURLConnection connection = (HttpURLConnection) url.openConnection();

    // Hack to force HttpURLConnection to run the request
    // Otherwise getErrorStream always returns null
    connection.getResponseCode();
    InputStream stream = connection.getErrorStream();
    if (stream == null) {
        stream = connection.getInputStream();
    }
    // This is a try with resources, Java 7+ only
    // If you use Java 6 or less, use a finally block instead
    try (Scanner scanner = new Scanner(stream)) {
        scanner.useDelimiter("\\Z");
        response = scanner.next();
    }
} catch (MalformedURLException e) {
    // Replace this with your exception handling
    e.printStackTrace();
} catch (IOException e) {
    // Replace this with your exception handling
    e.printStackTrace();
}

@slf You're right. It actually depends on the implementation and the only "official" way is to check if `getErrorStream` returns null, but that only works after forcing the request to be executed. I update my code to reflect this. — qwertzguy, Nov 11 '14 at 23:12

score 15 · Answer 3 · answered Jan 08 '11 at 10:19

try something like this:

try {
    String text = "url";
    URL url = new URL(text);
    URLConnection conn = url.openConnection();
    // fake request coming from browser
    conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB;     rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 3.5.30729)");
    BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
    String f = in.readLine();
    in.close();
    System.out.println(f);
} catch (Exception e) {
    e.printStackTrace();
}

score 5 · Answer 4 · edited May 23 '17 at 11:46

5

try this:

BufferedReader reader = new BufferedReader(new InputStreamReader(con.getResponseCode() / 100 == 2 ? con.getInputStream() : con.getErrorStream()));

source https://stackoverflow.com/a/30712213/505623

edited May 23 '17 at 11:46

Community

1
1

answered Nov 30 '15 at 06:27

Fathurrahman Amin

161
4
6

score 0 · Answer 5 · answered Apr 29 '16 at 21:18

I got the same error even after adding agent string. Finally after a days investigation figured out the issue. It is really weired if the url scheme start with "HTTPS" it results in error 403. It should be in lowercase ("https"). So make sure you call "url.toLowercase()" before opening the connection

HttpURLConnection reading response content on 403 error

5 Answers5

Linked