Http request returning different response in browser

Question

I'm getting a JSON object from a server, When I enter the following generated URL into my browser I get a response with "num_match": 18, however when running in my app I get a JSON object with "num_matches": 2.

The URL object is created like this

URL request;
request = new URL(url);

and connection like this:

HttpURLConnection connection = (HttpURLConnection) request.openConnection();
    connection.setConnectTimeout(MAX_TIME);
    connection.setReadTimeout(MAX_TIME);

url is a String and I am copying the string contents into my browser to test.

The string is:

http://search.3taps.com/?auth_token=xxxxxxxxxxxxxxxxxx&retvals=heading,body,timestamp,external_url,images,price&rpp=100&source=BKPGE|CRAIG|EBAYC|INDEE|KIJIJ&category=PWSM&radius=200mi&lat=26.244&long=-80.2&annotations={age:18 OR age:19 OR age:20 OR age:21 OR age:22}

The URL object has the following fields

query:

auth_token=xxxxxxxxxxxxxxxxxx&retvals=heading,body,timestamp,external_url,images,price&rpp=100&source=BKPGE|CRAIG|EBAYC|INDEE|KIJIJ&category=PWSM&radius=200mi&lat=26.244&long=-80.2&annotations={age:18 OR age:19 OR age:20 OR age:21 OR age:22}

file:

/?auth_token=xxxxxxxxxxxxxxxxxx&retvals=heading,body,timestamp,external_url,images,price&rpp=100&source=BKPGE|CRAIG|EBAYC|INDEE|KIJIJ&category=PWSM&radius=200mi&lat=26.244&long=-80.2&annotations={age:18 OR age:19 OR age:20 OR age:21 OR age:22}

host:

search.3taps.com

The response comes back as "success":true on both but with a discrepancy in the object returned. I don't know much about http, what could be causing this?

UPDATE: On further testing it seems like there is only a problem when the annotations entry is present

annotations={age:18 OR age:19 OR age:20 OR age:21 OR age:22}

seems to be causing the problem.

score 2 · Accepted Answer · edited May 23 '17 at 12:29

Make sure you are encoding the URL request correctly when you are setting the URL for the server. The spaces, braces, and colons all need to be appropriately escaped. Spaces should be %20, etc. This may help: HTTP URL Address Encoding in Java

Old Answer.... Comments indicate this does not affect the result... so moving down.

It is quite possible that the server is changing it's behaviour based on the type of 'browser' you are reporting yourself to be. When connecting to an HTTP server you tell the server what your UserAgent is (typically for a browser it is something like "Internet Explorer ...." or "Mozilla ..." or "Google Chome ...". The Server will often tailor the results of a request to suite the User Agent (different javascript files and HTML codes go to IE, etc.). This is also how servers re-direct mobile devices to a mobile-friendly version of a site.

It is quite possible that the server is changing it's response to match your UserAgent exposed by your Java code, (which by decault is something like "Java/1.7.0". You can change this value a few ways. Have a look at this question Setting user agent of a java URLConnection and try to run your program with the Mozilla agent, and see if you get different results.

I tried both of the methods from that link but still the same result. see the update to the original post. — Steve M, Aug 25 '13 at 23:53
Updated answer, check you are encoding the request properly. — rolfl, Aug 25 '13 at 23:58
Oh yes that must be it, the browser is adding the escape codes, will test in a few — Steve M, Aug 26 '13 at 00:04

Http request returning different response in browser

1 Answers1