5

I'm new to Javascript and PhantomJS, but my seemly simple objective has proven to be harder to achieve than expected. I want to write a script that would load a website, and then output the value of a Javascript variable used on that page. If I open that page in a browser and open the Javascript console, I can type in the variable name and it tells me the value associated to said variable. I just trying to reproduce this function but with PhantomJS so that I can automate this task.

Could someone point me towards the right documentation for this? I haven't been able to find how to do such a thing, assuming PhantomJS is the right way to do this. Maybe there is a simpler alternative?

Thanks.

Gleb Billig
  • 157
  • 3
  • 8
  • 1
    One of the following might help you: https://github.com/ariya/phantomjs/wiki/API-Reference-WebPage#wiki-webpage-evaluate or https://github.com/ariya/phantomjs/wiki/API-Reference-WebPage#onconsolemessage – Dexygen Sep 16 '13 at 15:05
  • Phantom JS is dead now. Use something in the canonical thread [How can I scrape pages with dynamic content using node.js?](https://stackoverflow.com/questions/28739098/how-can-i-scrape-pages-with-dynamic-content-using-node-js). – ggorlen Jan 04 '23 at 19:14

2 Answers2

16

What you need to understand is that phantomJS has two JavaScript environments and those two are independent of each other. The inner one is the document script (which you have in any browser). The outer one is controlling what phantomJS should do. It simulates the user.

So in a sense you need to tell phantomJS "the user opened the JavaScript console any typed ...". The evaluate command does this.

So to read the value of the variable foo, you write this code:

var foo = page.evaluate(function() {
    return document.foo;
});

Note: The document isn't strictly necessary but it helps to keep the two environments apart in the head of the developer.

Artjom B.
  • 61,146
  • 24
  • 125
  • 222
Aaron Digulla
  • 321,842
  • 108
  • 597
  • 820
  • I was able to get this to work, I had no idea you could evaluate the global variables on "the other side" how awesome! This whole time I was printing out the information onto tables on the screen and then physically evaluating the tables to get the information, this is so much more efficient! – William Howley Aug 21 '16 at 00:56
6

Aaron's answer didn't work for me. Maybe it was valid back in 2013, but now (2016), it doesn't work anymore with the current version. In Aaron's example, foo returns a promise, not a value. console.log(foo) outputs Promise { <pending> }.

Here's what I figured out :

page
    .evaluate( function(){ return window.foo })
    .then ( function(foo){ console.log "foo = ", foo});

Now you really get the value of foo.

Jeremy Thille
  • 26,047
  • 12
  • 43
  • 63