Why can't I set Java http User-Agent?

Question

I tried to set the User Agent for http request like this:

public BufferedReader readURL(String url){
        URL urlcon;
        BufferedReader in = null;
        try {
            urlcon = new URL(url);
            connection = (HttpURLConnection)urlcon.openConnection();

            System.setProperty("http.agent", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0)");
            connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0)");
            System.out.println(connection.getHeaderField("User-Agent"));
            connection.connect();
            in = new BufferedReader(
                                    new InputStreamReader(
                                        connection.getInputStream()));

            String header = connection.getHeaderField(0);
            System.out.println(header);
            System.out.println("---Start of headers---");
            int i = 1;
            while ((header = connection.getHeaderField(i)) != null) {
                String key = connection.getHeaderFieldKey(i);
                System.out.println(((key==null) ? "" : key + ": ") + header);
                i++;
            }
            System.out.println(connection.getHeaderField("http.agent"));
            System.out.println("---End of headers---");
        } catch (MalformedURLException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        return in;
    }

And what i got was User-Agent null:

null
HTTP/1.0 200 OK
---Start of headers---
Server: Apache
Cache-Control: max-age=10
Expires: Sun, 07 Aug 2011 16:09:26 GMT
Vary: Accept-Encoding
Content-Type: text/html
Content-Length: 163582
Date: Sun, 07 Aug 2011 16:09:20 GMT
X-Varnish: 889692780 889684459
Age: 4
Connection: keep-alive
X-Bip: 889692780 70 148
Via: 1.1 CachOS
null
---End of headers---

Why can't I set the User-Agent ?

possible duplicate of [How to modify the header of a HttpUrlConnection](http://stackoverflow.com/questions/480153/how-to-modify-the-header-of-a-httpurlconnection) — Romain Hippeau, Aug 07 '11 at 16:36
I've tryed to follow this other topic steeps but nothing working. I use JDK 1.6 update 24. — Felipe, Aug 07 '11 at 16:54

score 3 · Answer 1 · answered Aug 07 '11 at 16:18

3

Use setHeader(), not setRequestProperty.

answered Aug 07 '11 at 16:18

Tassos Bassoukos

16,017
2
36
40

5

Sorry but where I get this setHeader() ? I'm using URLConnection class. – Felipe Aug 07 '11 at 16:27

score 1 · Answer 2 · answered Jan 14 '17 at 22:44

Setting system property: "http.agent" will change your connection header: "User-Agent", but notice that according to documentation your java version is still written in it:

Misc HTTP properties

http.agent (default: “Java/”) Defines the string sent in the User-Agent request header in http requests.

Note that the string “Java/” will be appended to the one provided in the property (e.g. if -Dhttp.agent=”foobar” is used, the User-Agent header will contain “foobar Java/1.5.0” if the version of the VM is 1.5.0). This property is checked only once at startup.

Notice that property is checked only once at startup, so you may want to first set property, than create your first url and connect.

To verify that appropriate header is sent, you can use "tcpdump". Usage:

tcpdump -n dst host stackoverflow.com -vvvv

score 0 · Answer 3 · edited May 23 '17 at 12:16

0

The server returns the header information. I would guess "User-Agent" isn't important enough to be return. Just because it isn't returned doesn't mean it isn't sent.

I have a little tool (ieHTTPHeader) that displays the header information. When I do a refresh on this page this it the first set of headers that are sent and returned:

GET /questions/6973981/why-cant-i-set-java-http-user-agent HTTP/1.1
Accept: /
Referer: https://stackoverflow.com/questions/tagged/java?page=2&sort=newest&pagesize=15
Accept-Language: en-ca
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)
Accept-Encoding: gzip, deflate
Host: stackoverflow.com
Connection: Keep-Alive
Cookie: __utmc=140029553; __utma=140029553.1370458634.1310761265.1312727448.1312739618.123; __utmz=140029553.1312739618.123.123.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=yahoo%20finance%20api%20java; __qca=P0-1025379872-1310761265343; m=4; usr=t=cEyCYO7bXECF&s=X6DJTj5kuY8H; __utmb=140029553.15.10.1312739618

HTTP/1.1 200 OK
Cache-Control: public, max-age=60
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
Expires: Sun, 07 Aug 2011 18:25:43 GMT
Last-Modified: Sun, 07 Aug 2011 18:24:43 GMT
Vary: *
Date: Sun, 07 Aug 2011 18:24:42 GMT
Content-Length: 12040

edited May 23 '17 at 12:16

Community

1
1

answered Aug 07 '11 at 18:32

camickr

321,443
19
166
288

Thank you for your answer, but what is the right string that is the agent ? "http.agent" or "User-Agent" ? – Felipe Aug 07 '11 at 18:40
I got it.. thank you.. but in this case I'm setting the header.. should not I see it ? I tryed with other websites URLs and all same.. show the user agent as null. – Felipe Aug 07 '11 at 19:26
Did you read my response??? I already stated that I don't think you should see the user agent. I even gave you output of the headers when using IE. What more do you want? – camickr Aug 07 '11 at 19:39
Yes I read.. but who send the request is not me ? Who set the headers of the request is not me ? Sorry, maybe it's a newbie question but I would like your answer. Best regards. – Felipe Aug 07 '11 at 19:42
I will try with Apache http. :-/ – Felipe Aug 08 '11 at 16:41

Why can't I set Java http User-Agent?

3 Answers3