Getting HTML response instead of JSON in android

Question

I'm trying to retrieve data from mysql database located on a remote server through php into android. This worked fine while running in localhost. But when it is in remote server, I receive HTML codes instead of JSON response from server. I tried pasting the url (http://ksos.0fees.us/pgm_list.php?day=1&hall=A) in browser which resulted in correct JSON output. The HTML response is shown below.

<html><body>
<script type="text/javascript" src="/aes.js" ></script>
<script>function toNumbers(d)
 {var e=[];
  d.replace(/(..)/g,function(d){e.push(parseInt(d,16))});
  return e
 }
 function toHex()
  {for(var d=[],d=1==arguments.length&&arguments[0].constructor==Array?arguments[0]:arguments,e="",f=0;f<d.length;f++)
  e+=(16>d[f]?"0":"")+d[f].toString(16);
  return e.toLowerCase()
 }
 var a=toNumbers("f655ba9d09a112d4968c63579db590b4"),b=toNumbers("98344c2eee86c3994890592585b49f80"),c=toNumbers("5cb1c0309e553acda177d912f21ac485");
 document.cookie="__test="+toHex(slowAES.decrypt(c,2,a,b))+"; 
 expires=Thu, 31-Dec-37 23:55:55 GMT; 
 path=/";
 location.href="http://ksos.0fees.us/pgm_list.php?day=1&hall=A&ckattempt=1";
</script>
<noscript>This site requires Javascript to work, please enable Javascript in your browser or use a browser with Javascript support</noscript>
</body></html>

Following is my request to server for getting response

public String makeServiceCall(String url, int method, List<NameValuePair> params){
    try{
        DefaultHttpClient httpclient = new DefaultHttpClient();
        HttpEntity httpentity = null;
        HttpResponse httpresponse = null;

        if (method == POST) {
            HttpPost httppost = new HttpPost(url);
            httppost.setHeader("User-Agent", ua);
            httppost.setHeader("Accept", "application/json");
            if(params!=null){
                httppost.setEntity(new UrlEncodedFormEntity(params));
            }
            httpresponse = httpclient.execute(httppost);
        } else if(method == GET){
            if(params!=null){
                String paramString = URLEncodedUtils.format(params, "utf-8");
                url += "?" + paramString;
            }
            HttpGet httpget = new HttpGet(url);
//              HttpPost httppost = new HttpPost(url);
            httpget.setHeader("User-Agent", ua);
            httpget.setHeader("Accept", "application/json");
            httpresponse = httpclient.execute(httpget);
        }
        httpentity = httpresponse.getEntity();
        is = httpentity.getContent();
    }catch(UnsupportedEncodingException e){
        e.printStackTrace();
    }catch (ClientProtocolException e) {
         e.printStackTrace();
    }catch (IOException e) {
         e.printStackTrace();
    }
    try{
        BufferedReader reader = new BufferedReader(new InputStreamReader(is,"UTF-8"),8);
//          BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"),8);
        StringBuilder sb = new StringBuilder();
        String line = null;
        while ((line = reader.readLine())!= null) {
            sb.append(line+"\n");
        }
        is.close();
        response = sb.toString();
    }catch(Exception e){
        Log.e("Buffer Error", "Error : "+e.toString());
    }
    return response;
}

I've tried by setting and not setting setHeader for user agent. The php part looks as follows:

$q=mysql_query("SELECT ...........");
if(!empty($q)){
    while($row=mysql_fetch_assoc($q))
        $pgmlist[]=$row;
    $response["pgmlist"] = $pgmlist;
    echo json_encode($response);

}
else{
    $response["success"] = 0;
    $response["message"] = "No record found";
    echo json_encode($response);
}

Please post of the php code where you are echoing the output to your android application — Sourabh Kumar Sharma, Sep 05 '15 at 05:05
I don't expect any prob in php because I'm getting the desired output when I check with the URL in any browser — Niyaz, Sep 05 '15 at 05:28
Q: What part of this isn't a *server side* question? Q: Did you write (or control) the server side PHP application? Q: Does your localhost configuration use the *same* PHP app and the *same* HTTP server? Q: What do the server logs say? Any difference between the "localhost" and "remote" HTTP requests? How much troubleshooting have you done from the server side? — paulsm4, Sep 05 '15 at 05:29
The php given above is only server side. The localhost used the exactly the same things as used for remote server and gave the correct results. I've not done server side troubleshooting. What exactly should I do there? — Niyaz, Sep 05 '15 at 06:01
Your problem is almost certainly on the server side. One server sends back pure JSON (I'm guessing, based on your statement), the other sends back HTML + Javascript. You need to figure out *why the server is responding differently*. Again: Q: Are you running the *same* PHP app locally as the remote server is running ... or are you "emulating" it locally? Q: Do both local and remote have the same type of HTTP server/same PHP configuration? Q: Have you looked at the server logs (both local and remote)? Do you see the same "requests" in the logs? Q: Could authentication be an issue? — paulsm4, Sep 05 '15 at 06:35
PS: It sounds like the PHP snippet you provided, with the "select" and "json_encode()" *ISN'T EVEN BEING CALLED*. Q: What PHP occurs *before* this call? — paulsm4, Sep 05 '15 at 06:38
PPS: `mysql_query()` and friends are *OBSOLETE*. Please consider using [mysqli](http://php.net/manual/en/book.mysqli.php) or [PDO MySql](http://php.net/manual/en/ref.pdo-mysql.php). Look here for more info: http://stackoverflow.com/questions/12859942/why-shouldnt-i-use-mysql-functions-in-php — paulsm4, Sep 05 '15 at 06:42
Looks like something is injecting the HTML page. You should check it with your server provider, or with your cellular phone provider (try, for example, connecting your phone via WiFi and see if it works). Your code (however obsolete) is not the issue here. — uri2x, Sep 05 '15 at 07:19
Checked with WiFi. But no change. I know some lines of code are obsolete, but it works fine while running in localhost. — Niyaz, Sep 05 '15 at 10:55
Dude - find somebody who can look at the server side for you. Somebody needs to understand exactly what protocol the *server* is expecting - and exactly how it differs from whatever your localhost server is (or isn't) doing. It might be as simple as "the remote server is trying to establish a session" (by writing a cookie), or "the remote server needs authentication" (with some form of username/password). Please let us know what you find! — paulsm4, Sep 05 '15 at 21:05

score 4 · Answer 1 · answered Sep 11 '15 at 13:18

4

Atlast found solution...

The issue was not with the android or php part. It was the problem with the server. The hosting site which I used, sends cookies to client side which is not handled inside android but is automatically handled by browsers. I used to another hosting site where cookies are not involved and got the needed json output.

answered Sep 11 '15 at 13:18

Niyaz

81
1
8

2

Could you please explain more?? I have the same issue – Sujiz Sep 29 '15 at 05:57
2

Most of the free web hosting sites sent cookies for enhancing experience with the clients. This is fine in case of browsers as browsers manages with this cookies. But in an android app which we create, if we normally do not write any code to accept this cookies. Hence this situation comes. Either we will have to write code to accept cookies or else look for a server that doesnot send cookies. – Niyaz Oct 31 '15 at 13:41
Can you please explain and write code, how to accept cookies @Niyaz – Visal Varghese Mar 12 '17 at 08:22
what is the new hosting website that you used? – Samer Allahham Apr 09 '19 at 15:29

score 2 · Answer 2 · answered Mar 26 '18 at 19:56

I spent a long time to understand and solve the problem.

Firstly, we need to understand that 0fess hosting has anti bot technique which blocks the calls from (none-browser) clients. The main idea of this technique is using a javascript script that checks if the request is coming from a normal web browser then the script encrypts the IP and sets a cookie with key __test and value of the encrypted IP.

To solve such a problem we need to run a webview inside our application and request any page from the our 0fees site. then we can intercept the response and get the cookie. after that, we can use this cookie as a request header in our http rest requests.

This is a sample code

 WebView browser=new WebView(getContext());
    browser.getSettings().setJavaScriptEnabled(true); 
    browser.setWebViewClient(new WebViewClient() {
        @Override
        public void onPageFinished(WebView view, String url){
            final String cookies = CookieManager.getInstance().getCookie(url);
            Log.d("any", "All the cookies by me in a string:" + cookies);
            HttpPost httppost = new HttpPost("http://ksos.0fees.us/pgm_list.php");
            httppost.setHeader("Accept", "application/json");
            httppost.setHeader("Cookie", cookies);
            //continue your request paramters 
           }
       }
  );
browser.loadUrl("http://ksos.0fees.us/");

This answer gave me a good "north". I am using infinityfree.net and seeing the same behavior. Unfortunately, the solution here sometimes worked, sometimes it didn't. The server was doing something I just couldn't figure out. I moved to https://www.000webhost.com and now it works like a charm, no workaround needed. — Felipe Caldas, Sep 04 '19 at 10:59

Getting HTML response instead of JSON in android

2 Answers2

Linked