executing http POST returns HTML instead of JSON

Question

EDIT completely re-working the question for better understanding

I have to query the given url http://api.bf3stats.com/pc/player/ with 2 POST parameters: 'player' (for player name) and 'opt' (for options). I've tested it on http://www.requestmaker.com/ with following data: player=Zer0conf&opt=all. I'm getting a correct JSON response (thought I don't know how their site performs the query, I guess it's php). Now I'm trying to do the same in Android:

  private StringBuilder inputStreamToString(InputStream is) {
       //this method converts an inputStream to String representation
    String line = "";
    StringBuilder total = new StringBuilder();

    BufferedReader rd = new BufferedReader(new InputStreamReader(is));

    try {
        while ((line = rd.readLine()) != null) {
            total.append(line);
        }
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    return total;
}

and that's how I make the request:

 public void postData(String url, String name) {

    HttpClient httpclient = new DefaultHttpClient();
    HttpPost httppost = new HttpPost(url);

    try {

        List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
                    //qname is a String containing a correct player name
        nameValuePairs.add(new BasicNameValuePair("player", qname));
        nameValuePairs.add(new BasicNameValuePair("opt", "all"));
        httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));

        HttpResponse response = httpclient.execute(httppost);
        //test is just a string to check the result, which should be in JSON format
        test = inputStreamToString(response.getEntity().getContent())
                .toString();

    } catch (ClientProtocolException e) {

    } catch (IOException e) {

    }
}

What I'm getting in the 'test" String is not JSON, but the complete HTML-markup of some bf3stats page. What could be wrong with my request?

Answer 1

you need to set content type "application/x-www-form-urlencoded" in your request headers for the type of data you want to send.

I tested your API for the above mentioned data, it is working fine and i received loads of data in response. You can find it here.

You can try the following code:

public  String postDataToServer(String url) throws Throwable
    {

    HttpPost request = new HttpPost(url);
    StringBuilder sb=new StringBuilder();

    String requestData = prepareRequest();
    StringEntity entity = new StringEntity(requestData);
                         entity.setContentType("application/x-www-form-urlencoded;charset=UTF-8");//text/plain;charset=UTF-8
                         request.getParams().setParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, Boolean.FALSE);

                         request.setHeader("Accept", "application/json");
                         request.setEntity(entity); 
                         // Send request to WCF service 
                         HttpResponse response =null;
                         DefaultHttpClient httpClient = new DefaultHttpClient();
                         //DefaultHttpClient httpClient = getNewHttpClient();
                         HttpConnectionParams.setSoTimeout(httpClient.getParams(), 10*1000); 
                         HttpConnectionParams.setConnectionTimeout(httpClient.getParams(),10*1000); 
                         try{

                         response = httpClient.execute(request); 
                         }
                         catch(SocketException se)
                         {
                             Log.e("SocketException", se+"");
                             throw se;
                         }
                         /* Patch to prevent user from hitting the request twice by clicking 
                          * the view [button, link etc] twice.
                          */
                         finally{
                         }



    InputStream in = response.getEntity().getContent();
    BufferedReader reader = new BufferedReader(new InputStreamReader(in));
    String line = null;
    while((line = reader.readLine()) != null){
        sb.append(line);

    }
    Log.d("response in Post method", sb.toString()+"");
    return sb.toString();
    }

    public String prepareRequest()
    {

        return "player=Zer0conf&opt=all";
    }

Edit: I was getting error Expectation Failed error- 417 for your Webservice. So we need to add one more param in our request. It is request.getParams().setParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, Boolean.FALSE); . I have updated above code with this. Now it works fine.

Response looks something like this.

Hope this will help.

Cheers
N_JOY

Answer 2

Try to view the url in browser, I believe there is a !DOCTYPE declaration, which definitely cannot be parsed to a JSON object.