2

I am using Selenium with Chromedriver in Java (1.8) to do some automated web crawling:

System.setProperty("webdriver.chrome.driver", "chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("someurl.com");

I am attempting to migrate to an Ubuntu 16.04 Server. On the server I have installed the Ubuntu chromedriver version 2.37, chrome version 65. According to the chromedriver documentation these versions are compatible. I have changed the code specifying the location of the chromedriver in Ubuntu:

System.setProperty("webdriver.chrome.driver", "/usr/lib/local/chromedriver");

Prior to running my program I start up xvfb:

Xvfb -ac :99 -screen 0 1280x1024x16 &
export DISPLAY=:99

Then execute my program:

java -jar myprogram.jar

The following is printed to the console:

Starting ChromeDriver 2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7) on port 10574
Only local connections are allowed.
Exception in thread "main" org.openqa.selenium.WebDriverException: unknown error: Chrome failed to start: exited abnormally
  (Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Linux 4.4.0-112-generic x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 60.35 seconds
Build info: version: '3.11.0', revision: 'e59cfb3', time: '2018-03-11T20:33:15.31Z'
System info: host: 'ubuntu-s-1vcpu-1gb-nyc1-01', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-112-generic', java.version: '1.8.0_151'
Driver info: driver.version: ChromeDriver
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
        at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
        at org.openqa.selenium.remote.JsonWireProtocolResponse.lambda$new$0(JsonWireProtocolResponse.java:53)
        at org.openqa.selenium.remote.JsonWireProtocolResponse.lambda$getResponseFunction$2(JsonWireProtocolResponse.java:91)
        at org.openqa.selenium.remote.ProtocolHandshake.lambda$createSession$0(ProtocolHandshake.java:123)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)
        at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
        at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
        at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:464)
        at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:126)
        at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:73)
        at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:136)
        at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
        at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:545)
        at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:209)
        at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:132)
        at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:181)
        at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:168)
        at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:123)
        at RHio.Test.main(Test.java:39)
John Bartlett
  • 23
  • 1
  • 4
  • Chrome, starting from version 65, supports headless mode. No need for xvfb. – SiKing Mar 26 '18 at 22:32
  • Ok, I have tried using the headless ChromeOption, and not running xvfb, but the problem persists... – John Bartlett Mar 26 '18 at 23:03
  • Works for me. Without additional information, there could be a hundred different things wrong. Have a read through how to create [mcve]. – SiKing Mar 26 '18 at 23:10
  • What versions of chromedriver & chrome do you use? I suspect it is something wrong with my installation based on related questions I've found – John Bartlett Mar 26 '18 at 23:25
  • I don't even know! I always use the latest. I use this this do it for me: https://github.com/prashant-ramcharan/webdriver-binary-downloader – SiKing Mar 26 '18 at 23:29
  • 1
    perhaps a stupid question, but do you have Chrome installed? (not just chromedriver, but the actual browser) I'm guessing you don't. – Corey Goldberg Mar 27 '18 at 18:28
  • I do have chrome installed, as noted above its pretty brand new, version 65... @SiKing thanks for that link I will try reinstalling with that – John Bartlett Mar 27 '18 at 19:43

1 Answers1

4

The error does gives us some idea about whats wrong happening :

Exception in thread "main" org.openqa.selenium.WebDriverException: unknown error: Chrome failed to start: exited abnormally

As you are trying to initiate Chromedriver and Chrome on Ubuntu Server headless you have to pass a couple of Options as follows :

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--headless")
options.add_argument("--no-sandbox")
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(chrome_options=options, executable_path='/path/to/chromedriver')
driver.get('http://google.com/')
driver.quit()

Additional Notes

As per the mandatory Requirements and #1341 ensure the following :

  • Chromedriver is within /usr/local/bin/
  • Chrome Browser is within /usr/bin/google-chrome

tl;dr

The story of Sandbox

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • 1
    Most of those options have nothing to do with headless Chrome. In Java you can just pass `chromeOptions.setHeadless(true);` and I am sure Python has something similar. Also if you pass `Chrome(chrome_options=options, executable_path='/path/to/chromedriver')` why do you also have to ensure "Chromedriver is within /usr/local/bin/"? – SiKing Mar 27 '18 at 20:03
  • 1
    After reinstalling both chromedriver and chrome, making sure they were in those folders, and adding all of those chromeOptions arguments it worked! Thanks – John Bartlett Mar 27 '18 at 20:26
  • 1
    In my case it seems that the --no-sandbox did the trick. Thanks! – silver_mx Mar 29 '18 at 14:30
  • 1
    Thanks @DebanjanB!! – silver_mx Mar 31 '18 at 08:20
  • `Chromedriver is within /usr/local/bin/` - seems not necessary any more as you can specify it in right in the code e.g. `System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver_linux64_v81/chromedriver");` - worked for me – yetanothercoder May 06 '20 at 16:04