3

I am trying to do a GET request with some parameters in Java using HttpURLConnection. Everytime I do this however, I get a 400: Bad Request each time. What do I need to change to make it work?

String url = "http://www.awebsite.com/apath?p1=v1&p2=v2&p3=v3";
HttpURLConnection conn = (HttpURLConnection)new URL(url).openConnection();
conn.setDoInput(true);
conn.setDoOutput(false);
conn.setUseCaches(false);
conn.setRequestMethod("GET");
conn.setRequestProperty("Host", "www.awebsite.com");
conn.setRequestProperty("User-Agent", "Mozilla/4.0");
conn.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-us,en;q=0.5");
conn.setRequestProperty("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
conn.setRequestProperty("Keep-Alive", "115");
conn.setRequestProperty("Connection", "keep-alive");
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuilder data = new StringBuilder();
String s = "";
while((s = br.readLine()) != null)
    data.append(s);
String pageData = data.toString();

I have tried:

  • Using URLEncoder on the whole query (after the ?) and just on the values.
  • Setting the content length header.
  • Setting the connection to use output and putting the query as the output.
DanielGibbs
  • 9,910
  • 11
  • 76
  • 121
  • Are you writing your own bot? Why are you cloning Mozilla? – Buhake Sindi Jun 14 '11 at 09:34
  • I switched the URL to http://google.com and it worked fine for me. – hoipolloi Jun 14 '11 at 09:34
  • @The Elite Gentleman I used Mozilla/4.0 for the User-Agent as I read in the cURL documentation that it is needed for some CGIs. @hoipolloi Weird, the context I am using it in has a rather long query and that doesn't work. – DanielGibbs Jun 14 '11 at 09:44
  • @Daniel: Were you able to fix this issue, I have a similar scenario, I am getting a 200 response if my url connection fires a short query, but I get a 400 when the query is long. – user620339 Apr 05 '12 at 19:29
  • @user620339 No, I didn't fix it but I did come to a conclusion as to the cause: http://stackoverflow.com/questions/6341602/httpurlconnection-get-request-getting-400-bad-request#comment7419861_6341669 – DanielGibbs Apr 09 '12 at 06:56

2 Answers2

2

It seems that in this case the HTTP 400 is being used not to signify an error in the request syntax, but a logical error as described here: HTTP 400 (bad request) for logical error, not malformed request syntax.

Community
  • 1
  • 1
DanielGibbs
  • 9,910
  • 11
  • 76
  • 121
2

The code attempts to open a connection to www.awebsite.com, but it also sends illegal/invalid values for the Host field: www.google.com. This is definitely not allowed by the HTTP specification.

You would have to correct this, to ensure that the server at www.awebsite.com receives the correct set of headers, so that it can process your request.

Obligatory link: How to use java.net.URLConnection to fire and handle HTTP requests?

Community
  • 1
  • 1
Vineet Reynolds
  • 76,006
  • 17
  • 150
  • 174
  • 1
    Oops my bad, the URL was originally a very long one with google as the host, I simplified it for the example. – DanielGibbs Jun 14 '11 at 09:41
  • @DanieL, even then I don't think the HTTP request would be well formed. I suspect there would be two `Host` headers in the request. I think you should remove the line involving the `Host` header. – Vineet Reynolds Jun 14 '11 at 09:43
  • I have tried removing the `Host` header but no change. I also used the code in the link you posted but I get the same error. – DanielGibbs Jun 14 '11 at 09:52
  • Is it possible for you to share the source so that I could take a look? If you can however, get a network capture of the traffic in Wireshark and compare it with a browser issued request. – Vineet Reynolds Jun 14 '11 at 09:54
  • 1
    @DanieL, ignore my previous point if you haven't found out the cause of the problem. Can you verify if you problem goes away when you write your code in the manner depicted in the section "Firing a HTTP GET request with (optionally) query parameters" of the link provided? Apparently, I don't think it's just the Host header. It could also be the fact that you are setting the request method, which might be redundant as well. – Vineet Reynolds Jun 14 '11 at 10:05
  • Following the code in "Firing a HTTP GET request with (optionally) query parameters" produces the same error. I am comparing them in WireShark and they are more or less identical. The only difference is that my headers are in a slightly different order. – DanielGibbs Jun 14 '11 at 10:09
  • Can you please post both the Wireshark captures as text in your question? It might just help. – Vineet Reynolds Jun 14 '11 at 10:11
  • Sorry, I can't actually do that as the content of the request is confidential. I think I'm going to have to come to the conclusion that the 400 is being used to signify not a syntax error in the request, but as a logical error as described here: http://stackoverflow.com/questions/4781187/http-400-bad-request-for-logical-error-not-malformed-request-syntax. Thanks for all your help though! – DanielGibbs Jun 14 '11 at 10:16
  • Alright, you're welcome. But, if you do need to verify if the header orders are causing the problem (unlikely but possible), then use a HTTP proxy like Fiddler or WebScarab to change the order. – Vineet Reynolds Jun 14 '11 at 10:19