PhantomJS doesn't manipulate dropdown with select tag

Question

Been having some trouble with some phantomjs code I'm trying to run. In theory, the code should be able to select an option from the below dropdown menu.

<select name="Budget_Ctr2" size="1" onchange="JavaScript:Submitthis('Budget_Ctr2')" id="Budget_Ctr2">
  <option selected="selected" value="CAAH">CAAH</option>
      <option value="CAFLS">CAFLS</option>
      <option value="CECAS">CECAS</option>

All of the code works, except when I get to the following snippet:

var sel = document.getElementById('Budget_Ctr2');
var opts = sel.options;

Everything I've read on stack so far has made me think this should work. However, when I run the program, I get "TypeError: null is not an object (evaluating 'sel.options'). I'm fairly confident 'sel' itself isn't null based on a few test, but I can't reconcile the error with what's going on. Not super experienced with javascript outside of phantomjs, but anyone have any ideas?

Doing this in a console I get values `(3) [option, option, option, selectedIndex: 0]`. — user3366016, Oct 27 '17 at 18:06

score 0 · Answer 1 · answered Oct 28 '17 at 10:12

Why is null not an object? It doesn't exist in the DOM when the PhantomJS script is run.

But why is this element visible in dev tools of non-headless browser then? It was probably added programmatically, as a result of work of a script.

Remember: if you run page.evaluate as soon as page.open callback fires, it is run very quickly - page scripts probably haven't done their jobs yet. So you can either wait an approximate time, and then page.evaluate like this:

var seconds = 3;
page.open(function(){

    setTimeout(function(){

        page.evaluate(){
            // do stuff
        }

    }, 1000 * seconds);

});

Or you can wait for the target element to appear, see waitfor.js

score -1 · Answer 2 · answered Oct 27 '17 at 20:21

It seems that your script is placed in the head section. If so, then it is executed before the body element is loaded. Try to run console.log(document.body); command and it will return null.

You need to wrap your code in a window.onload handler or include it inside $(document).ready() in jQuery, or place it before the </body> tag.

Please see this answers to get more information.

PhantomJS doesn't manipulate dropdown with select tag

2 Answers2