Is it possible to run capybara-webkit (i.e. forked webkit_server) on Heroku Cedar?

Question

I need to run capybara-webkit inside a Rails application to enable headless web browsing with JavaScript support (i.e. not for testing/CI purposes, and webrat or other acceptance testing drivers/frameworks will not work). I'm wondering if this is possible on a Heroku deployment, specifically because it requires QtWebKit and the ability to fork the webkit_server process with socket communication. I'm open to creative ideas on how to make this work on Heroku (e.g. a pool of worker dynos). I'm hoping someone has a better handle on what constraints exist in the Heroku environment, or can categorically rule out the possibility so I can move on to AWS EC2 if necessary.

Searching for this tends to turn up a lot about capybara testing and add-ons for CI servers, neither of which are relevant for my use case. I'm not testing anything (at least not in the traditional cucumber/rspec/etc sense) - I'm using Capybara's integration with the webkit driver, finders and node/element model to navigate a website that requires a significant amount of client-side JS in order to work.

I'm also open to other (native Ruby) solutions for programmatically interacting with web sites using JavaScript-enabled DOM.

Answer 1

I spoke to Heroku support about this and their answer was that this is basically a) unsupported; b) very difficult, including (among other things) a statically built version of QtWebKit.

My own investigation into this on Amazon EC2 also made me realize that QtWebKit requires a running instance of Xvfb. I highly doubt this would be available on Heroku, and I suspect it would be extremely difficult to make it work.

My own approach has been to put this functionality on an EC2 instance. After making some attempts with Amazon's standard AMIs (their build and RHEL), I found that the packages available through Ubuntu's package management systems made it MUCH easier to get up an running.

Long story short: Heroku is a non-starter, Amazon EC2 with Ubuntu is the best way to go.

Answer 2

I was able to successfully run Capybara + Poltergeist + PhantomJS on Heroku

I've placed compiled phantomjs binaries for OSX (for my development machine) and linux-64 (for Heroku) in bin/ folder of my Rails application.

initializers/capybara.rb

require 'capybara/poltergeist'

Capybara.register_driver :poltergeist do |app|
  phantomjs_path = if RUBY_PLATFORM['x86_64-darwin']
                     Rails.root.join('bin', 'phantomjs-osx').to_s
                   elsif RUBY_PLATFORM['x86_64-linux']
                     Rails.root.join('bin', 'phantomjs-linux-64').to_s
                   else
                     raise "Can't load PhantomJS for OS: #{RUBY_PLATFORM}"
                   end

  options = {
      phantomjs: phantomjs_path,
      phantomjs_logger: Logger.new('/dev/null'),
      phantomjs_options: %w[--load-images=no --ignore-ssl-errors=yes],
      js_errors: false,
      timeout: 90
  }
  Capybara::Poltergeist::Driver.new(app, options)
end

Capybara.default_driver = :poltergeist
Capybara.javascript_driver = :poltergeist
Capybara.default_wait_time = 1

Example code:

session ||= Capybara::Session.new(:poltergeist)
session.visit('http://google.com')

Good luck!

Answer 3

You may be able to accomplish what you want using PhantomJS.

This project has a JavaScript, rather than Ruby API, although the browser instance can expose a web-server, allowing you to communicate with it from Ruby over HTTP.

http://code.google.com/p/phantomjs/wiki/Interface