0

I have a Ruby (non-Rails) & Watir web scraper working locally, however after I deployed it on Heroku, it looks like Heroku is unable to start Chrome/Chromedriver. I tried following the solutions from here: Heroku: Unable to find chromedriver when using Selenium

Here is my configuration:

args = %w[--disable-infobars  --no-sandbox --disable-gpu]
options = {
       binary: ENV['GOOGLE_CHROME_BIN'],
       prefs: { password_manager_enable: false, credentials_enable_service: false },
       args:  args
     }
b = Watir::Browser.new(:chrome, options: options)

Gemfile

ruby '2.7.1'

gem 'watir', '6.19.0'
gem 'xpath', '3.2.0'
gem 'google-api-client'
gem 'webdrivers'

edit And set the following buildpacks:

heroku buildpacks:set https://github.com/heroku/heroku-buildpack-google-chrome
heroku buildpacks:set https://github.com/heroku/heroku-buildpack-chromedriver

And here is the error message in the Heroku log:

unknown error: Chrome failed to start: exited abnormally. (Selenium::WebDriver::Error::UnknownError)
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /app/.apt/opt/google/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

I also ran it with the args set to --headless

args = %w[--disable-infobars --headless window-size=1600,1200 --no-sandbox --disable-gpu]

and got this message:

timed out after 30 seconds, waiting for true condition on #<Watir::Browser:0x7a2c669409432a9a url="https://client.schwab.com/login/signon/customercenterlogin.aspx" title="Login | Charles Schwab"> (Watir::Wait::TimeoutError)

I had originally set the config vars to the following as I was following Heroku: Unable to find chromedriver when using Selenium:

heroku config:set GOOGLE_CHROME_BIN=/app/.apt/opt/google/chrome/chrome
heroku config:set GOOGLE_CHROME_SHIM=/app/.apt/opt/google/chrome/chrome

But I realized it was pointing to the wrong path, when I ran this in Heroku console find /app/ -name "*chrome*", the path I had set didn't exist, and I found the new path: /app/.apt/usr/bin/google-chrome

/app/.apt/usr/bin/google-chrome
/app/.apt/usr/bin/google-chrome-stable
/app/.apt/usr/share/gnome-control-center/default-apps/google-chrome.xml
/app/.apt/usr/share/man/man1/google-chrome-stable.1.gz
/app/.apt/usr/share/man/man1/google-chrome.1.gz
/app/.apt/usr/share/doc/google-chrome-stable
/app/.apt/usr/share/applications/google-chrome.desktop
/app/.apt/usr/share/menu/google-chrome.menu
/app/.apt/usr/share/appdata/google-chrome.appdata.xml
/app/.apt/opt/google/chrome
/app/.apt/opt/google/chrome/chrome_100_percent.pak
/app/.apt/opt/google/chrome/chrome
/app/.apt/opt/google/chrome/google-chrome
/app/.apt/opt/google/chrome/cron/google-chrome
/app/.apt/opt/google/chrome/chrome-sandbox
/app/.apt/opt/google/chrome/chrome_200_percent.pak
/app/.apt/etc/cron.daily/google-chrome
/app/.profile.d/010_google-chrome.sh
/app/.profile.d/chromedriver.sh
/app/.chromedriver
/app/.chromedriver/bin/chromedriver
/app/vendor/bundle/ruby/2.7.0/gems/webdrivers-4.6.0/lib/webdrivers/tasks/chromedriver.rake
/app/vendor/bundle/ruby/2.7.0/gems/webdrivers-4.6.0/lib/webdrivers/chrome_finder.rb
/app/vendor/bundle/ruby/2.7.0/gems/webdrivers-4.6.0/lib/webdrivers/chromedriver.rb
/app/vendor/bundle/ruby/2.7.0/gems/webdrivers-4.6.0/spec/webdrivers/chrome_finder_spec.rb
/app/vendor/bundle/ruby/2.7.0/gems/webdrivers-4.6.0/spec/webdrivers/chromedriver_spec.rb
/app/vendor/bundle/ruby/2.7.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome.rb
/app/vendor/bundle/ruby/2.7.0/gems/selenium-webdriver-3.142.7/lib/selenium/webdriver/chrome
/app/vendor/bundle/ruby/2.7.0/gems/google-api-client-0.53.0/generated/google/apis/chromeuxreport_v1
/app/vendor/bundle/ruby/2.7.0/gems/google-api-client-0.53.0/generated/google/apis/chromeuxreport_v1.rb

So I then changed my vars to:

heroku config:set GOOGLE_CHROME_BIN=/app/.apt/usr/bin/google-chrome
heroku config:set GOOGLE_CHROME_SHIM=/app/.apt/usr/bin/google-chrome

My questions:

  1. Is my config vars GOOGLE_CHROME_BIN & GOOGLE_CHROME_SHIM pointing to the right files now? or should they be pointing to different bin files?

  2. It looks like Heroku is still not detecting the updated config vars, as the log is still showing /app/.apt/opt/google/chrome/chrome. After I updated them, I've pushed the changes to Heroku, triggering a rebuild, and also resetted the dynos. Even in the config vars settings in dashboard, it is showing the updated vars. What else can I do to make Heroku recognize the updated config vars?

Thanks for any help!

MLZ
  • 403
  • 7
  • 20
  • Try the instructions here: https://github.com/titusfortner/webdrivers/wiki/Heroku-buildpack-google-chrome – titusfortner Mar 26 '21 at 21:56
  • @titusfortner Since this is a standalone Ruby project (no rails), I will try option 2, or can I try option 1 as well?. Does the line: driver = Selenium::WebDriver.for :chrome, replace my line: b = Watir::Browser.new(:chrome, options: options)? Please keep in mind this is not for a test, I'm using Watir as a web scraper. – MLZ Mar 27 '21 at 03:20
  • I have changed the shim var to /app/.apt/usr/bin/google-chrome-stable, now I get the message: /app/vendor/bundle/ruby/2.7.0/gems/webdrivers-4.6.0/lib/webdrivers/chrome_finder.rb:21:in `location': Failed to find Chrome binary. (Webdrivers::BrowserNotFound) – MLZ Mar 27 '21 at 03:28
  • Watir uses Selenium under the hood, so variables set on Selenium classes will be used by Watir the same way, and it doesn't matter why you are using the tool. Did you follow the note: "heroku-buildpack-chromedriver should be uninstalled to avoid any conflicts with this gem." – titusfortner Mar 27 '21 at 03:34

0 Answers0