Get request works with POSTMAN but throws garbage response in my Java application

Question

New in Java, I am trying to fetch data from the following URL; https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY

I can fetch using POSTMAN without any request header while when I am trying through HttpsURLConnection it's throwing as

java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)

Code :

private void testIt() {
        String https_url = "https://www.nseindia.com/api/option-chain-indices?symbol=NIFTY";
        URL url;
        try {
            url = new URL(https_url);
            HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
            BufferedReader br = new BufferedReader(new InputStreamReader(con.getInputStream()));
            String input;
            while ((input = br.readLine()) != null) {
                System.out.println(input);
            }
            br.close();
        } catch (MalformedURLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

After adding lots of headers as;

  con.setRequestProperty("method", "GET");
    con.setRequestProperty("authority", "www.nseindia.com");
    con.setRequestProperty("scheme", "https");
    con.setRequestProperty("path", "/api/option-chain-indices?symbol=NIFTY");
    con.setRequestProperty("user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36");
    con.setRequestProperty("sec-fetch-dest", "empty");
    con.setRequestProperty("accept", "application/json");
    con.setRequestProperty("sec-fetch-site", "same-origin");
    con.setRequestProperty("sec-fetch-mode", "cors");
    con.setRequestProperty("referer","https://www.nseindia.com/get-quotes/derivatives?symbol=NIFTY&identifier=OPTIDXNIFTY05-03-2020CE11500.00");
    con.setRequestProperty("accept-encoding", "gzip, deflate, br");
    con.setRequestProperty("accept-language", "en-US,en;q=0.9,hi;q=0.8,es;q=0.7");
    con.setRequestProperty("if-none-match", "W/\"dfaed-ueAny3lZCxK1ooMpkoUm3OMoPag");

But receiving garbage values rather than JSON which I have received through POSTMAN as;

áÑ>^5?T[RãVI÷¤œ;täR^Êj•vW¯`‡1í’'…é½ßß¬E=ÝõL#M™„ë¬ê÷AÌÊ›=«¦:*Wáó~O|V•AOq3"/ÃC°T¤B¾2–½ÌW%LÐ‹â^SÑ^§árG`¦?`$²‚sL›¹ÕÖ)·Ð_J¹¥(?Ûºxœ»ÔÖ)·Üž/s©Íè´]ÀRQûøÊX~¡Km?Ë?™¶?žª¬ñ“
L§—BÑH…C†sÝ•ö}¤;ÐšX«¶xFçûSš\w¥†Ç¹É?Ør»W¹’ýr.ì®[dg?ïâ?èçGQ†^¿úÐ¿úž3'go¾í•á¡Œ¼£öÊ,#Ã jHì…Î$0×É¯,‡‡|’%²ôe+DN%ó›÷‰UŒæm« xŒ,Ú@êŠ€ÊÿlçSwý®Oƒpgs—#áu6÷¬²Jgß–¢Î€I–gi3<ÃâÇèöÎÎº-ŽÔ,y´àDr*VÖÇ}?Qà'¥Ü™
Þ:%

https://stackoverflow.com/questions/17079273/java-net-sockettimeoutexception-read-timed-out-under-tomcat--- you can check the timeout settings — Brooklyn99, Mar 02 '20 at 20:30
Time Out could be caused by the server is waiting for input, like the body in a POST request. I recommend using a library with higher-level like HttpClient or OkHttp — David SK, Mar 02 '20 at 21:33

kometen · Accepted Answer · 2020-03-03T21:49:41.060

2

Try add requestmethod and user-agent.

try {
    HttpURLConnection con = (HttpURLConnection) url.openConnection();
    con.setRequestMethod("GET");
    con.setRequestProperty("User-Agent", USER_AGENT);
    int response_code = con.getResponseCode();

    if (response_code == 200) {
        BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
        String input_line;
        StringBuffer response = new StringBuffer();
        while ((input_line = in.readLine()) != null) {
            response.append(input_line);
        }
        in.close();
    }
}

edited Mar 03 '20 at 21:49

answered Mar 02 '20 at 20:43

kometen

6,536
6
41
51

Thanks for the response. I have added headers but received garbage responses. – Somu Mar 02 '20 at 21:25
NP. I tried to connect again using both java and postman but it may have been blocked from the internet. – kometen Mar 02 '20 at 21:55

score 0 · Answer 2 · edited Jul 11 '20 at 22:19

0

After removing below header it worked for me,

con.setRequestProperty("accept-encoding", "gzip, deflate, br");

edited Jul 11 '20 at 22:19

krohit

792
1
7
26

answered Jul 11 '20 at 18:30

Karthikeyan Selvaraj

1
1

I think you should explain why this happens, as I see that @Somu is confused about [server-side compression](https://en.wikipedia.org/wiki/HTTP_compression) techniques – Vipul Panth Jul 11 '20 at 19:26

Get request works with POSTMAN but throws garbage response in my Java application

2 Answers2