0

I wrote this bit of code to be called in a script that runs on a cron job, knowing it has to be a headless browser to be ran in a cron job, i found Headless. It sounds like a wonderful gem to do exactly what i want it do to, the only problem is it still opens up FireFox when I run the code.

I thought the whole point of headless was to not have to access the display and run in the background, like :phantomjs. Am I missing something or did I mistake what the headless gem is supposed to accomplish? (P.S. at bottom about when i tried to use :phantomjs)

#encoding: utf-8
require 'watir-webdriver'
require 'headless'

#log into admin dashboard
headless = Headless.new
browser = Watir::Browser.start 'http://app.mycompany.com/admin'
browser.link(:xpath =>'/html/body/div/div/div/div/a').when_present.click
browser.text_field(:id => 'Email').when_present.set 'me@mycompany.com'
browser.button(:id => 'next').click
browser.text_field(:id => 'Passwd').when_present.set 'password'
browser.button(:id => 'signIn').click
browser.goto 'https://app.mycompany.com/admin/dashboard'


#browser is at dashboard to grab yesterday's numbers 
code that grabs data

#closes browser after grabbing data
browser.close
headless.destroy

#send timestamp
current_time = Time.now

puts "Screen grabbed at " + current_time.inspect + "\n\n"

#puts all data into array then outputs array split on each metric's title
dailyreportdata = [my glorious array of data]

dailyreportdata.each_slice(2) { |x|
  puts x.join
}

My script runs to completion, but the data doesnt appear so I am guessing that it fails over when it tries to load the browser, thus loading no data to grab and send to my file.

My script look like this:

#!/bin/sh


_now=$(date +"%m_%d_%Y")


ruby dailyreportscraper.rb > ~/dailyscrape_$_now.txt

if I run it outside of the cron job, it works just fine.

P.S. - I tried phantomjs but every time it got to the "Enter Email" field, it would time out on waiting for the element to appear - it is a google login so maybe there is something to do with that, I even tried using the xpath as well.

Thanks for the help!

imadirtycunit
  • 125
  • 1
  • 10
  • Did you install a virtual framebuffer (`sudo apt-get install xvfb`)? – glenn jackman Oct 15 '15 at 22:01
  • i am pretty new to all this so bare with me, i am on mac so i tried to brew install xvfb, but it doesnt know what xvfb is.....i installed the x11 qaurtz stuff but can't firgure out how it get it to work.... – imadirtycunit Oct 16 '15 at 20:06
  • google search for "xvfb osx" gives this as first result: http://stackoverflow.com/questions/25451133/xvfb-run-on-os-x – glenn jackman Oct 16 '15 at 20:10
  • i tried to install quartz to use the xvfb but it does not work. perhaps i could revisit the phantomjs perspective.... – imadirtycunit Oct 16 '15 at 21:24

1 Answers1

1

Call headless.start after you do headless = Headless.new.

And make sure you are running Xvfb.

Mike S
  • 11,329
  • 6
  • 41
  • 76
  • good point. I tried that too, it still opens FF....i suppose i might have installed Xvfb wrong or something, i cannot figure out how to run it.... – imadirtycunit Oct 16 '15 at 20:07