passing a python object into a casperjs script iterating over the object and returning a result object to python

Question

I'm new to programing in languages more suited to the web, but I have programmed in vba for excel.

What I would like to do is:

1. pass a list (in python) to a casper.js script.
2. Inside the casperjs script I would like to iterate over the python object (a list of search terms)
3. In the casper script I would like to query google for search terms
4. Once queried I would like to store the results of these queries in an array, which I concatenate together while iterating over the python object.
5. Then once I have searched for all the search-terms and found results I would like to return the RESULTS array to python, so I can further manipulate the data.

QUESTION --> I'm not sure how to write the python function to pass an object to casper.

QUESTION --> I'm also not sure how to write the casper function to pass an javascript object back to python.

Here is my python code.

import os
import subprocess
scriptType = 'casperScript.js'
APP_ROOT = os.path.dirname(os.path.realpath(__file__))
PHANTOM = '\casperjs\bin\casperjs'
SCRIPT = os.path.join(APP_ROOT, test.js)
params = [PHANTOM, SCRIPT]
subprocess.check_output(params)

js CODE

var casper = require('casper').create();
casper.start('http://google.com/', function() {
this.echo(this.getTitle());
});
casper.run();

Answer 1

Could you use JSON to send the data to the script and then decode it when you get it back?

Python:

json = json.dumps(stuff) //Turn object into string to pass to js

Load a json file into python:

with open(location + '/module_status.json') as data_file:
    data = json.load(data_file);

Deserialize a json string to an object in python

Javascript:

arr = JSON.parse(json) //Turn a json string to a js array json = JSON.stringify(arr) //Turn an array to a json string ready to send to python

Answer 2

You can use two temporary files, one for input and the other for output of the casperjs script. woverton's answer is ok, but lacks a little detail. It is better to explicitly dump your JSON into a file than trying to parse the console messages from casperjs as they can be interleaved with debug strings and such.

In python:

import tempfile
import json
import os
import subprocess

APP_ROOT = os.path.dirname(os.path.realpath(__file__))
PHANTOM = '\casperjs\bin\casperjs'
SCRIPT = os.path.join(APP_ROOT, test.js)

input = tempfile.NamedTemporaryFile(mode="w", delete=False)
output = tempfile.NamedTemporaryFile(mode="r", delete=False)

yourObj = {"someKey": "someData"}
yourJSON = json.dumps(yourObj)
input.file.write(yourJSON)
# you need to close the temporary input and output file because casperjs does operations on them
input.file.close()
input = None
output.file.close()

print "yourJSON", yourJSON

# pass only file names
params = [PHANTOM, SCRIPT, input.name, output.name]
subprocess.check_output(params)

# you need to open the temporary output file again
output = open(output.name, "r")
yourReturnedJSON = json.load(output)
print "returned", yourReturnedJSON
output.close()
output = None

At the end, the temporary files will be automatically deleted when the objects are garbage collected.

In casperjs:

var casper = require('casper').create();
var fs = require("fs");

var input = casper.cli.raw.get(0);
var output = casper.cli.raw.get(1);
input = JSON.parse(fs.read(input));
input.casper = "done"; // add another property
input = JSON.stringify(input);
fs.write(output, input, "w"); // input written to output
casper.exit();

The casperjs script isn't doing anything useful. It just writes the inputfile to the output file with an added property.