6

Using HttpClient 4.1.3, I've written the following code:

HttpClient httpClient = HttpClientFactory.newHttpClient();
HttpGet httpGet = new HttpGet("some/url/to/hit");
HttpResponse httpResp = httpClient.execute(httpGet);
int statusCode = httpResp.getStatusLine().getStatusCode();
if(statusCode != HttpStatus.SC_OK)
    throw new Exception(/* ... */);

That's getting HTTP 500 responses (as found in the httpResp.getStatusLine().getStatusCode()) from a particular URL and throwing the exception.

The thing is, when I go to the "failing" URL in a browser, its running perfectly fine.

So I ask:

  • Could HttpClient be timing out, short-circuiting the request-response cycle, and just giving me an HTTP 500?
  • What else could be going on here? How is it possible for HttpClient to be giving me 500s when the browser is displaying the page perfectly fine for the same exact URL?

Thanks in advance!

IAmYourFaja
  • 55,468
  • 181
  • 466
  • 756
  • Try to debug the request from the web browser. Maybe what causing your problem is that you are not setting the headers expected by the server. – dlock Oct 18 '12 at 12:15
  • What server are you hitting? Turn on request logging to see your incoming request. Do you have access to error logs on the server? If it is returning a 500, there should be logs. – jalynn2 Oct 18 '12 at 12:24
  • @jalynn2 - how would I turn on request logging? Is this an HttpClient construct or are you talking about something like Firebug's console? – IAmYourFaja Oct 18 '12 at 12:28
  • I mean on the web server. What type of server is it? – jalynn2 Oct 18 '12 at 12:29
  • It's a Java/Spring backend hosting a product by another team. I can access the logs by talking to them (and them, in turn, looking at the logs for me!) but was hoping to rule out client-side issues before taking that approach. – IAmYourFaja Oct 18 '12 at 12:34
  • I pulled down httpcomponents-4.1.3 source and spent some time trying to see what conditions HttpClient claims HTTP 500 results under. I haven't gotten very far as its a pretty big codebase. Is it possible the server is timing out? Does anybody know how HttpClient determines 500 status? – IAmYourFaja Oct 18 '12 at 12:35
  • The status should be coming back from the server. 500 means "internal server error". I don't know the internals of HttpClient, but I would be very surprised if it gave a 500 on timeout. – jalynn2 Oct 18 '12 at 12:39
  • You can [instruct](http://hc.apache.org/httpcomponents-client-ga/logging.html) the HTTP client to log the request and response (or just the headers) and see what's going on. You can do the same at the browser and see the differences. Hint: some servers try to guess the browser from the headers and react differently depending on that. – ivant Oct 18 '12 at 13:29
  • You can also try to inspect the HTTP Messages with Wireshark and diff the requests from your app and the browser. Can you give us the actual url you are using? Maybe it's a charcter that should better be url-encoded? – Fildor Oct 18 '12 at 15:28

2 Answers2

2

I had the same problem. I could access a web site via the browser but when using the apache http client, I consistently got http 500 internal server error. The problem was the "Content-Type" GET Header. It had the value "multipart/related",which some servers don't seem to like. I changed it to "text/html" and it all worked fine.

Hope this helps.

cristian.petroaca
  • 355
  • 2
  • 4
  • 13
2

I had the same problem accessing a spring xml based api. The problem solved by setting the accept header to xml and html.

httpGet.setHeader("Accept", "text/html,application/xhtml+xml,application/xml;");
Stelios Joseph Karras
  • 463
  • 1
  • 7
  • 18