8

Question : I would like to know if it's possible to do parallel browsing with the testing framework in one script file, so with the tester module and casperjs test command.

I've seen some people create two casper instances : CasperJS simultaneous requests and https://groups.google.com/forum/#!topic/casperjs/Scx4Cjqp7hE , but as said in the doc, we can't create new casper instance in a test script.

So i tried doing something similar-simple example- with a casper testing script (just copy and execute this it will work):

var url1 = "http://casperjs.readthedocs.org/en/latest/testing.html"
    ,url2 = "http://casperjs.readthedocs.org/en/latest/testing.html"
    ;

var casperActions = {
    process1: function () {
        casper.test.begin('\n********* First processus with our test suite : ***********\n', function suite(test) {
            "use strict";
            casper.start()
            .thenOpen(url1,function(){
                this.echo("1","INFO");
            });
            casper.wait(10000,function(){
                casper.test.comment("If parallel, it won't be printed before comment of the second processus !");
            })
            .run(function() {
                this.test.comment('----------------------- First processus over ------------------------\n');
                test.done();
            });
        });
    },
  process2: function () {
        casper.test.begin('\n********* Second processus with our test suite : ***********\n', function suite(test) {
            "use strict";
            casper.start()
            .thenOpen(url1,function(){
                this.echo("2","INFO");
            });
            casper.test.comment("Hi, if parallel, i'm first !");
            casper.run(function() {
                this.test.comment('----------------------- Second processus over ------------------------\n');
                test.done();
            });
        });
    }
};

['process1', 'process2'].forEach(function(href) {
    casperActions[href]();
});

But it's not parallel, they are executed one by one. Currently, i do some parallel browsing but with node so not in the file itself, using child process. So if you split my previous code in two files -proc1.js,proc2.js- (just the two scenarios->casper.test.begin{...}), and launch the code below via node, something like that will work-with Linux, i have to search the equivalent syntax for windows- :

var exec = require("child_process").exec
;

exec('casperjs test proc1.js',function(err,stdout,stderr){
console.log('stdout: ' + stdout);
console.log('endprocess1');
});
exec('casperjs test proc2.js',function(err,stdout,stderr){
console.log('stdout: ' + stdout);
console.log('endprocess2');
});

My problem is that the redirections and open new urls is quite long, so i want for some of them being execute in parallel. I could do XXX files and launch them in parallel with node, but i don't want XXX files with 5 lines of code, so if someone succeeded (if it's possible) to open urls in parrallel in the same testing file without node (so without multiple processes), please teach me!

And i would like to know what is the difference between chaining instructions, or re-use the casper object each time :

so between that :

casper.test.begin('\n********* First processus with our test suite : ***********\n', function suite(test) {
    "use strict";
    casper.start()
    .thenOpen(url1,function(){
        this.echo("1","INFO");
    })
    .wait(10000,function(){
        casper.test.comment("If parallel, it won't be print before comment of the second processus !");
    })
    .run(function() {
        this.test.comment('----------------------- First processus over ------------------------\n');
        test.done();
    });
});

And that :

casper.test.begin('\n********* First processus with our test suite : ***********\n', function suite(test) {
    "use strict";
    casper.start();
    casper.thenOpen(url1,function(){
        this.echo("1","INFO");
    });
    casper.wait(10000,function(){
        casper.test.comment("If parallel, it won't be print before comment of the second processus !");
    })
    casper.run(function() {
        this.test.comment('----------------------- First processus over ------------------------\n');
        test.done();
    });
});

Chaining my instructions, will it block all the chain if one of my step fail (promise rejected) instead of executing every casper steps?

So it would be better to chaining instructions with dependant steps [like thenClick(selector)] and use the casper object with independant steps (like open a new url), wouldn't it?

Edit : i tried and if a step fail, chained or not, it will stop all the next steps, so i don't see the difference using chained steps or not...

Community
  • 1
  • 1
Fanch
  • 3,274
  • 3
  • 20
  • 51
  • A promise rejection is like a `throw`, it will skip the rest of the chain until it is handled. – Benjamin Gruenbaum Mar 19 '14 at 17:42
  • And in casperjs, do you know when the promise is fulfilled? When a step is completed? Because for example, a test which fails won't block the chain, just the closure. – Fanch Mar 19 '14 at 17:55
  • I didn't know that casperjs has promises implementation ? Btw, if all you want is to run tests in parallel then there are grunt and gulp tasks that do that for you – rusln Apr 30 '14 at 17:47
  • "the callback/listener stuff is an implementation of the Promise pattern." Yes, grunt does it, but we finally choose to do it manually. (node child process) + promises-es6. – Fanch May 02 '14 at 07:34

2 Answers2

3

Well, chaining or using the casper object each time is just a matter of taste, it does the same, and we can't launch several instances of casper in a testing script. If you have a loop which opens some links, you'll have to wait for each page to be loaded sequentially.

To launch parallel browsing with the testing framework, you have to do multiple processes, so using node does the trick.

After digging, I finally split files with too many redirections to be not longer than my main scenario which can't be split. A folder with 15 files is executed -parallel- in 2/4 min, on local machine.

Fanch
  • 3,274
  • 3
  • 20
  • 51
  • 1
    Any details on the multiple process with nodeJS to launch casperJS ? Aren't you facing the memory leak problem of parallelizing phantomJS ? It would detail the answer to the original question. Thanks very much – bdavidxyz Aug 26 '14 at 08:11
  • 2
    Better late than never : no I haven't faced the memory leak problem of parallelizing phantomJS, we have used promises ( promises-es6 module) and a custom node module to set the maximum of casper processus at once (depending of the server capacities). So when we executed 75 tests; there was for example 15 scenarii running at the same time and other in a queue waiting the previous one to release a place. – Fanch Apr 22 '15 at 15:59
  • seems great, would gladly see a gist. But if I understood correctly, you still have to write down files by yourself for each test. it would be great to pass a string of js instead of a filename to casper – Rayjax Sep 15 '15 at 10:27
  • I'm not sure I understand what you said , if I answer : "we used a json config file to set the folder(s)/test(s) to execute, to easily configure jenkins builds", do I answer your thoughts ? That way it looked for every test files using `fs.readdir`. – Fanch Sep 15 '15 at 11:55
  • So you can put test files in a folder and they are auto found and tested. This is great if you have manually defined test suites which you write into test files, but what I need is to execute dynamically generated casper files (this used for scraping mostly, with some page interaction). As the parameters of the 'test' file are generated upon each scraper request, I would need to write them down to a temp file, send it to casper, and destroy it afterwards. That is why I would have loved a way to send js scripts to casper without having to write them to FS (as we can do with phantomJS directly) – Rayjax Sep 15 '15 at 12:20
  • What do you mean by "generated casper files" ? -invoking dynamic then() steps related to parameters- ? , do you have a crawler (using another tool that casper?) getting some page infos like links, etc... and clicking everywhere ? Can't you write this parameters in json tmp file and use them in casper? – Fanch Sep 15 '15 at 13:40
1

There's no official support for parallel browsing right now in casperjs, There are multiple work arounds I've set up several different environments and am about to test witch one is the best

I see one person is working with multiple casper instances this way. 

var google = require('casper').create(); 
var yahoo = require('casper').create(); 

google.start('http://google.com/'); 

yahoo.start('http://yahoo.com/', function() { 
    this.echo(google.getTitle()); 
}); 

google.run(function() {}); 

yahoo.run(function() {}); 

setTimeout(function() { 
    yahoo.exit(); 
}, 5000);

Currently I am running multiple caspers in node using 'child_process'. It is very heave on both CPU and memory

Peter the Russian
  • 1,170
  • 8
  • 9