I'm attempting to do some web scraping using headless chrome via selinium-webdriver on Heroku but most recently ran into trouble. Visiting, https://music.youtube.com I get a page with the following:
<div class="message">
Sorry, YouTube Music is not optimized for your browser. Check for updates or try Google Chrome.
</div>
I can confirm that my chrome driver is up to date and nothing on my end has changed to cause this break in functionality. Using any other automated scraping gem similar to selinium gives me a similar message in telling me my browser is deprecated. Note that performing these actions is no problem when I do it on my local machine. To give some background, I originally followed this answer to get everything running correctly on heroku prior to the breakage. On top of this here's my setup:
gem 'selenium-webdriver'
gem 'webdrivers', '~> 4.0', require: false
require 'webdrivers'
require 'selenium-webdriver'
chrome_bin_path = ENV.fetch('GOOGLE_CHROME_SHIM', nil)
options = Selenium::WebDriver::Chrome::Options.new
options.binary = chrome_bin_path if chrome_bin_path
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-gpu')
$driver = Selenium::WebDriver.for :chrome, options: options
What would be the most logical answer to why I'm getting this message on a site like youtube?
