Getting "Set-Cookie" header

Question

I'm trying to get 'Set Cookie' header using apache httpclietn-4.2.2 and having some problems.

Header in Firebug:

Set-Cookie  remixreg_sid=deleted; expires=Thu, 10-Nov-2011 04:32:30 GMT; path=/; 
domain=.vk.com remixapi_sid=deleted; expires=Thu, 10-Nov-2011 04:32:30 GMT; path=/; 
domain=.vk.com remixrec_sid=deleted; expires=Thu, 10-Nov-2011 04:32:30 GMT; path=/;
domain=.vk.com remixsid=0000000000000000000000000000000000000000000000000000; expires=Mon, 04-Nov-2013 16:10:24 GMT; path=/; domain=.vk.com

How I'm trying to obtain it:

 //location is a header with url I need to do GET request to
 Header location = response.getFirstHeader("Location");
 HttpGet httpGet = new HttpGet(location.getValue());
 httpClient.getParams().setParameter(
 //tried to use different policies
 ClientPNames.COOKIE_POLICY, CookiePolicy.RFC_2965);
 Header [] allHeaders=response.getAllHeaders();

In allHeaders I have all headers except "Set Cookie". And I have warnings like this:

WARNING: Invalid cookie header: "Set-Cookie: remixlang=0; expires=Mon, 18-Nov-2013 
03:21:47 GMT; path=/; domain=.vk.com". Unrecognized cookie header 'Set-Cookie: 
remixlang=0; expires=Mon, 18-Nov-2013 03:21:47 GMT; path=/; domain=.vk.com'
Nov 09, 2012 4:31:41 AM org.apache.http.client.protocol.ResponseProcessCookies 
processCookies

So I think the problem is with 'expires' date.

What I tried to do:

1) Invalid cookie header : Unable to parse expires attribute when expires attribute is empty Created custom CookieSpec and tried to use it:

 httpClient.getCookieSpecs().register("vkCookie", new CookieSpecFactory() {
     public CookieSpec newInstance(HttpParams params){
         return new VkCookieSpec();
         }
     });
HttpClientParams.setCookiePolicy(httpClient.getParams(), "vkCookie");

2) Tried to set Data Format in httpClient params :

  httpClient.getParams().setParameter(CookieSpecPNames.DATE_PATTERNS, Arrays.asList("EEE, dd-MMM-yyyy HH:mm:ss z"));

But I'm still getting that warning. Would appreciate any help.

score 8 · Answer 1 · answered Sep 14 '16 at 08:15

I know this is an old question. But I had the same problem and just wanted to post my snippet to solve it, in particular setting CookieSpecs.STANDARD explicitly (see spec on apache commons for details):

        RequestConfig globalConfig = RequestConfig.custom()
                .setCookieSpec(CookieSpecs.DEFAULT)
                .build();
        CloseableHttpClient httpClient = HttpClients.custom()
                .setDefaultRequestConfig(globalConfig)
                .build();
        RequestConfig localConfig = RequestConfig.copy(globalConfig)
                .setCookieSpec(CookieSpecs.STANDARD)
                .build();
        HttpGet httpGet = new HttpGet(url);
        httpGet.setConfig(localConfig); 

        // Request
        CloseableHttpResponse response = httpClient.execute(httpGet);

Hope this helps.

score 6 · Accepted Answer · edited Apr 09 '17 at 03:12

6

You are trying to parse 'Set-Cookie' header with the RFC 2965 compliant spec, whereas RFC 2965 accepts 'Set-Cookie2' headers only.
The cookie in question is malformed. It contains non-standard 'expires' attribute, which, to make matters worse, contains a reserved character (comma) without enclosing quote marks. However, given it is a very common protocol violation HttpClient should be able to parse this cookie using 'best_match', 'browser_compatibility' or 'netscape_draft' policies.

In fact, one should always be using the 'best_match' policy and let HttpClient pick up the best matching policy based on the composition of the cookie headers.

edited Apr 09 '17 at 03:12

Onema

7,331
12
66
102

answered Nov 09 '12 at 09:59

ok2c

26,450
5
63
71

Thanks for the reply. I tried to use all the policies available. When I use best match, browser_compatibility or netscape_draft, there is no Set-Cookie header in the responce and no warnings appeared. – pomkine Nov 09 '12 at 10:34
@pomkine if there is no Set-Cookie header in the response, how do you expect HttpClient to be able to parse it? – ok2c Nov 09 '12 at 10:47
ohhh, sorry. I mean there is no Set-Cookie headers returned by response.getAllHeaders(). – pomkine Nov 09 '12 at 11:02
@pomkine response#getAllHeaders() always returns all non-malformed headers exactly as received from the wire. – ok2c Nov 09 '12 at 11:16
So if it is malformed I should get a warning?hm, but I'm getting no warnings and got all the headers except Set-Cookie when I set policy to "best_match". – pomkine Nov 09 '12 at 11:35
@pomkine If HttpClient encounters a malformed header of any sort, it will throw a protocol exception and terminate the connection. This behaviour can be overridden, though. I suspect the problem you are having has nothing to do with cookies in the first place. – ok2c Nov 09 '12 at 11:52
Any ideas what can be the problem? :) – pomkine Nov 09 '12 at 11:57
@pomkine Post _complete_ wire / context log of the session. See http://hc.apache.org/httpcomponents-client-ga/logging.html – ok2c Nov 09 '12 at 12:03
Thx for the hint with logging :) All I needed is to disable automatic redirect. Now I got my cookies. – pomkine Nov 13 '12 at 17:35
BEST_MATCH and BROWSER_COMPATIBILITY are deprecated in favor of DEFAULT. But DEFAULTS sift the warnings, only NETSCAPE remains. – Zon Nov 01 '16 at 15:03

Getting "Set-Cookie" header

2 Answers2