28

this is a question about how to persist cookies from one casperjs page to another..

so basically i got a nodejs file that spawns casperjs as a worker to do certain tasks.. one is to login, once logged in I store the cookie in a file.

when i spawn the next casper worker.. i want it to to use the cookie rather having to login again.. both these methods failed:

first: when i spawn the worker capserjs I add the --cookies-file=./cookiefilename ie var child = spawn('casperjs',['scrape.js','--cookies-file=./'+cookieFileName]);

second: within the casperjs worker file.. I make it read and set the cookie from a file ie

var casper = require('casper').create();
var cookieFileName = 'monsterCookie.txt';

// grab cookies from file 
var fs = require('fs');
var utils = require('utils');
var cookies = fs.read(cookieFileName);

casper.page.setCookies(cookies); 

casper.start('domain/page.html', function() {
    //FAIL! cookies aren't used here
    this.debugHTML();
});

casper.run();

notes:

  1. it was mentioned earlier that start removes cookies from the page? if so how do I prevent that?
  2. I know that sessions persist within the same phantomjs page object (see here https://gist.github.com/abbood/5347252) and same happens within the same casperjs page object (see here https://gist.github.com/abbood/5347287)
  3. keep in mind that I store cookies as is in the file (ie without any json/cookie parsing at all).. so my cookie file looks exactly like this

[General] cookies="@Variant(\0\0\0\x7f\0\0\0\x16QList\0\0\0\0\x1\0\0\0\n\0\0\0YCNTR=LB; expires=Tue, 09-Apr-2013 17:12:05 GMT; domain=.recruiter.domain.com; path=/\0\0\0qUID=13eb22f-2.21.171.120-1365523938; expires=Mon, 30-Mar-2015 16:12:18 GMT; domain=.domain.com; path=/\0\0\0]UIDR=1365523938; expires=Mon, 30-Mar-2015 16:12:18 GMT; domain=.domain.com; path=/\0\0\0[R_LANG=en; expires=Thu, 09-May-2013 16:16:06 GMT; domain=.recruiter.domain.com; path=/\0\0\0\x94\x43=4gpUmUGr2jgDrs4xOJVrGaNbD8DtYSd1E6quyLhe3E4F3EAGhbRJucnDgRVDeHh0; expires=Thu, 09-May-2013 16:16:06 GMT; domain=.recruiter.domain.com; path=/\0\0\0\x94WT_FPC=id=20cf093f17f2c6f3d041365495136954:lv=1365495369854:ss=1365495136954; expires=Fri, 07-Apr-2023 08:16:09 GMT; domain=.domain.com; path=/\0\0\0\xc4\x41\x43OOKIE=C8ctADE3OC4xMzUuMTQ3LjM5LTI4NzQ5NzQ0LjMwMjkxMjYxAAAAAAAAAAABAAAAmyoBAMo+ZFHhPWRRAQAAAAJWAADKPmRR4T1kUQAAAAA-; expires=Thu, 09-Apr-2015 16:16:10 GMT; domain=statse.domain.com; path=/\0\0\0Yv1st=CE061E87215F2D73; expires=Wed, 19-Feb-2020 14:28:00 GMT; domain=.domain.com; path=/\0\0\0\x84\x43OOKIE_ID=178.135.147.39-2368749744.30291261; expires=Fri, 07-Apr-2023 16:16:11 GMT; domain=cookie.domain.com; path=/DCS000065_7K5I\0\0\0\xbe\x41\x43OOKIE=C8ctADE3OC4xMzUuMTQ3LjM5LTIzNjg3NDk3NDQuMzAyOTEyNjEAAAAAAAABAAAAQQAAAM0+ZFHNPmRRAQAAAAEAAADNPmRRzT5kUQAAAAA-; expires=Fri, 07-Apr-2023 16:16:13 GMT; domain=cookie.domain.com; path=/)"

abbood
  • 23,101
  • 16
  • 132
  • 246
  • Sounds like this _might_ be a bug. Could be worth posting on the [CasperJS Google Groups page](https://groups.google.com/forum/#!forum/casperjs) if you haven't already. – thealexbaron Apr 10 '13 at 04:56
  • Also, are you sure that the cookies are successfully being read and parsed? – thealexbaron Apr 10 '13 at 04:58
  • @thealexbaron I did [bring](https://github.com/n1k0/casperjs/issues/175) up the issue with Niko, the founder of Casper, still waiting for his response.. but that's a good point you made about making sure the cookie is successfully read and parsed.. i'm not exactly sure (off the top of my head) how to do that.. lemme know if you got suggestions.. but i'll dig deeper into that – abbood Apr 10 '13 at 05:31
  • talked to the creator of casperjs.. asked me to open a new issue.. issue created [here](https://github.com/n1k0/casperjs/issues/445) – abbood Apr 10 '13 at 07:54
  • Just do console.log(cookies) after var cookies = fs.read(cookieFileName); -- show us the output. – thealexbaron Apr 10 '13 at 16:18

2 Answers2

66

Saving cookies:

var fs = require('fs');
var cookies = JSON.stringify(phantom.cookies);
fs.write(cookieFilename, cookies, 644);

Restoring cookies:

var fs = require('fs');
var data = fs.read(cookieFilename);
phantom.cookies = JSON.parse(data);

The phantom is global variable in PhantomJS. More information you can get in wiki

Artjom B.
  • 61,146
  • 24
  • 125
  • 222
iimos
  • 4,767
  • 2
  • 33
  • 35
  • 1
    ugh, why the F doesn't this work by passing the damn flag. THANK YOU for this solution. – Sean Clark Dec 04 '15 at 05:13
  • 2
    Login if cookies example on Stack Overflow: https://github.com/cirosantilli/stack-overflow-vote-fraud-script/blob/1477d09ae365c2997ded514539898efd55171091/vote.js Notes: 1) `phantom` is available by default from CasperJS, no need to require it. 2) `fs` is part of PhantomJS as well: http://phantomjs.org/api/fs/method/is-file.html Unchangeable pre-node stuff: http://stackoverflow.com/questions/15745394/can-phantomjs-work-with-node-js – Ciro Santilli OurBigBook.com Dec 25 '15 at 01:09
10

@imos Great answer! It just helped me out and I was hoping I could add some more.

You can also add individual cookies to a page using:

var fs = require('fs')
var data = fs.read(file)
var cookies = JSON.parse(data)

for(var i = 0; i < cookies.length; i++) {
    phantom.addCookie(cookies[i]);
}

This might help if there are multiple cookie files.

Ryguy
  • 266
  • 2
  • 9