Python selenium - Not working in Ubuntu

Question

Below is the code snippet of what I am running on Ubuntu 16.04.

    from selenium import webdriver
    from selenium.webdriver.chrome.options import Options
    path_to_chromedriver = '/usr/local/bin/chromedriver' # change path as needed
    options = webdriver.ChromeOptions()
    options.add_argument("start-maximized")
    options.add_argument('--disable-extensions')
    options.add_argument('--headless')
    options.add_argument('--disable-gpu')
    options.add_argument('--no-sandbox') 
    options.add_argument('--no-sandbox')
    service_log_path = "{}/chromedriver.log".format('/tmp/local/')
    service_args = ['--verbose']
    driver = webdriver.Chrome(executable_path = path_to_chromedriver,chrome_options=options,
            service_args=service_args,
            service_log_path=service_log_path)

Doesnt matter what flag I put, I always get below error in chromedriver.log file.

[1.001][INFO]: COMMAND InitSession {
   "capabilities": {
      "alwaysMatch": {
         "browserName": "chrome",
         "goog:chromeOptions": {
            "args": [ "--headless", "--no-sandbox" ],
            "extensions": [  ]
         },
         "platformName": "any"
      },
      "firstMatch": [ {

      } ]
   },
   "desiredCapabilities": {
      "browserName": "chrome",
      "goog:chromeOptions": {
         "args": [ "--headless", "--no-sandbox" ],
         "extensions": [  ]
      },
      "platform": "ANY",
      "version": ""
   }
}
[1.001][INFO]: Populating Preferences file: {
   "alternate_error_pages": {
      "enabled": false
   },
   "autofill": {
      "enabled": false
   },
   "browser": {
      "check_default_browser": false
   },
   "distribution": {
      "import_bookmarks": false,
      "import_history": false,
      "import_search_engine": false,
      "make_chrome_default_for_user": false,
      "show_welcome_page": false,
      "skip_first_run_ui": true
   },
   "dns_prefetching": {
      "enabled": false
   },
   "profile": {
      "content_settings": {
         "pattern_pairs": {
            "https://*,*": {
               "media-stream": {
                  "audio": "Default",
                  "video": "Default"
               }
            }
         }
      },
      "default_content_settings": {
         "geolocation": 1,
         "mouselock": 1,
         "notifications": 1,
         "popups": 1,
         "ppapi-broker": 1
      },
      "password_manager_enabled": false
   },
   "safebrowsing": {
      "enabled": false
   },
   "search": {
      "suggest_enabled": false
   },
   "translate": {
      "enabled": false
   }
}
[1.001][INFO]: Populating Local State file: {
   "background_mode": {
      "enabled": false
   },
   "ssl": {
      "rev_checking": {
         "enabled": false
      }
   }
}
[1.001][INFO]: Launching chrome: /opt/google/chrome/google-chrome --disable-background-networking --disable-client-side-phishing-detection --disable-component-update --disable-default-apps --disable-hang-monitor --disable-prompt-on-repost --disable-sync --disable-web-resources --enable-logging --full-memory-crash-report --ignore-certificate-errors --load-extension=/tmp/.com.google.Chrome.np7aOC/internal --logging-level=1 --metrics-recording-only --no-first-run --password-store=basic --remote-debugging-port=12652 --safebrowsing-disable-auto-update --safebrowsing-disable-download-protection --use-mock-keychain --user-data-dir=/tmp/.com.google.Chrome.iUR4hY data:,
[1.001][DEBUG]: DevTools request: http://127.0.0.1:12652/json/version
[1.002][WARNING]: PAC support disabled because there is no system implementation
[927:927:0316/063553.173763:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

[1.037][DEBUG]: DevTools request failed
[1.087][DEBUG]: DevTools request: http://127.0.0.1:12710/json/version
[1.087][DEBUG]: DevTools request failed
[1.138][DEBUG]: DevTools request: http://127.0.0.1:12710/json/version
................................................................
................................................................
[61.031][INFO]: RESPONSE InitSession unknown error: Chrome failed to start: exited abnormally
[61.031][DEBUG]: Log type 'driver' lost 0 entries on destruction
[61.031][DEBUG]: Log type 'browser' lost 0 entries on destruction

I tried to run it from non root user as well, but still, I get same error.

I can only SSH to this machine. I tried, if running the command in screen can help but no use, nor does using crontab.

Any guidance will be very helpful.

UPDATE 1:

Below is the chromedriver log after reboot:

[1.003][INFO]: COMMAND InitSession {
   "capabilities": {
      "alwaysMatch": {
         "browserName": "chrome",
         "goog:chromeOptions": {
            "args": [ "start-maximized", "--disable-extensions", "--headless", "--disable-gpu", "--no-sandbox" ],
            "extensions": [  ]
         },
         "platformName": "any"
      },
      "firstMatch": [ {

      } ]
   },
   "desiredCapabilities": {
      "browserName": "chrome",
      "goog:chromeOptions": {
         "args": [ "start-maximized", "--disable-extensions", "--headless", "--disable-gpu", "--no-sandbox" ],
         "extensions": [  ]
      },
      "platform": "ANY",
      "version": ""
   }
}
[1.003][INFO]: Populating Preferences file: {
   "alternate_error_pages": {
      "enabled": false
   },
   "autofill": {
      "enabled": false
   },
   "browser": {
      "check_default_browser": false
   },
   "distribution": {
      "import_bookmarks": false,
      "import_history": false,
      "import_search_engine": false,
      "make_chrome_default_for_user": false,
      "show_welcome_page": false,
      "skip_first_run_ui": true
   },
   "dns_prefetching": {
      "enabled": false
   },
   "profile": {
      "content_settings": {
         "pattern_pairs": {
            "https://*,*": {
               "media-stream": {
                  "audio": "Default",
                  "video": "Default"
               }
            }
         }
      },
      "default_content_settings": {
         "geolocation": 1,
         "mouselock": 1,
         "notifications": 1,
         "popups": 1,
         "ppapi-broker": 1
      },
      "password_manager_enabled": false
   },
   "safebrowsing": {
      "enabled": false
   },
   "search": {
      "suggest_enabled": false
   },
   "translate": {
      "enabled": false
   }
}
[1.003][INFO]: Populating Local State file: {
   "background_mode": {
      "enabled": false
   },
   "ssl": {
      "rev_checking": {
         "enabled": false
      }
   }
}
[1.004][INFO]: Launching chrome: /opt/google/chrome/google-chrome --disable-background-networking --disable-client-side-phishing-detection --disable-component-update --disable-default-apps --disable-hang-monitor --disable-prompt-on-repost --disable-sync --disable-web-resources --enable-logging --full-memory-crash-report --ignore-certificate-errors --load-extension=/tmp/.com.google.Chrome.wpOJGk/internal --logging-level=1 --metrics-recording-only --no-first-run --password-store=basic --remote-debugging-port=12725 --safebrowsing-disable-auto-update --safebrowsing-disable-download-protection --use-mock-keychain --user-data-dir=/tmp/.com.google.Chrome.XeSkzL data:,
[1.004][DEBUG]: DevTools request: http://127.0.0.1:12725/json/version
[1.008][WARNING]: PAC support disabled because there is no system implementation
[1752:1752:0321/084842.042837:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
[1.051][DEBUG]: DevTools request failed
[1.102][DEBUG]: DevTools request: http://127.0.0.1:12725/json/version
[1.102][DEBUG]: DevTools request failed
[1.152][DEBUG]: DevTools request: http://127.0.0.1:12725/json/version
[1.152][DEBUG]: DevTools request failed
[1.203][DEBUG]: DevTools request: http://127.0.0.1:12725/json/version

undetected Selenium · Answer 1 · 2018-03-29T06:42:46.540

The error says it all :

[455:455:0316/061123.770013:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
[61.031][INFO]: RESPONSE InitSession unknown error: Chrome failed to start: exited abnormally

The error stack trace clearly mentions you were executing the program as a root user without --no-sandbox argument.

Solution

Follow the following steps :

Login as a non root user.
Copy chromedriver binary in a more accessible location (e.g. '/usr/local/bin/') and provide the necessary permissions (chmod 777)

Use the following code block :

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

path_to_chromedriver = '/path/to/chromedriver'
options = Options()
options.add_argument("start-maximized")
options.add_argument("--headless")
options.add_argument("--disable-gpu")
options.add_argument("--disable-extensions")
options.add_argument("--no-sandbox") 
service_args = ['--verbose']
service_log_path = '/tmp/local/chromedriver.log'
driver = webdriver.Chrome(executable_path='/path/to/chromedriver', chrome_options=options, service_args=service_args, service_log_path=service_log_path)
driver.get('http://google.com/')

Trivia

The story of Sandbox

Update A

As you are still seeing the error as :

[927:927:0316/063553.173763:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

Try the following steps :

Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
Use CCleaner tool to wipe off all the OS chores before and after the execution of your Test Suite.
If your base Web Client version is too old, then uninstall it through Revo Uninstaller and install a recent GA and released version of Web Client.
Execute your @Test.

Update B

As per the mandatory Requirements and #1341 ensure the following :

Chromedriver is within /usr/local/bin/
Chrome Browser is within /usr/bin/google-chrome

About the [1.008][WARNING]: PAC support disabled you can safely ignore the log as it is hard-coded in Chromium Browser Engine as per the discussion PAC support disabled because there is no system implementation

Update C

As per the Update section in the discussion How to make firefox headless programatically in Selenium with python? can you try the set_headless(headless=True) method as an alternative as follows :

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.set_headless(headless=True)
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\path\to\chromedriver.exe')
driver.get("http://google.com/")
print ("Headless Chrome Initialized")
driver.quit()

Additionally have a look at the discussion Running Chromedriver on Ubuntu Server headlessly

As I said, I tried with non root user as well. I tried the code above with another user but it is still giving same error. — Pratibha, Mar 16 '18 at 14:33
@pratibha Can you update the question with your updated code block and updated error stack trace for further analysis? — undetected Selenium, Mar 16 '18 at 14:39
I can still see `[927:927:0316/063553.173763:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.`. Can you perform a `cleanup` as per my answer update? — undetected Selenium, Mar 19 '18 at 09:20
Its a Ubuntu machine and I have access via SSH only. How can I do cleanup there? — Pratibha, Mar 19 '18 at 10:15
I have updated the question with output of chromedriver log. — Pratibha, Mar 21 '18 at 12:53
Yes both files are there, /usr/bin/google-chrome after few redirections point to /opt/google/chrome/google-chrome — Pratibha, Mar 21 '18 at 13:31
@pratibha I suppose that's where your program is shows the error. Either change install location or create a direct link. — undetected Selenium, Mar 21 '18 at 13:37
I tried that but still same error. I am using non root user as well. I switch user via `su - newuser` — Pratibha, Mar 21 '18 at 14:23
@pratibha Can you update the question with your current code trial and the complete error stack trace? — undetected Selenium, Apr 01 '18 at 17:18