8

I was reading over the Selenium docs and i couldn't quite work out whether you could run it on a server - as part of a larger web app.

ie. something happens in the web app that triggers the automated Selenium script to run, then returns result to web app.

Is that a possible use of Selenium or does it always need an actual browser to run the tasks inside of, or can it run a virtual browser for itself ?

sam
  • 9,486
  • 36
  • 109
  • 160

3 Answers3

16

Selenium always need an instance of a browser to control.

Luckily, there are browsers out there that aren't that heavy as the usual browsers you know. You don't have to open IE / Firefox / Chrome / Opera. You can use HtmlUnitDriver which controls HTMLUnit - a headless Java browser that does not have any UI. Or a PhantomJsDriver which drives PhantomJS - another headless browser running on WebKit.

Those headless browsers are much less memory-heavy, usually are faster (since they don't have to render anything), they don't require a graphical interface to be available for the computer they run at and are therefore easily usable server-side.

Petr Janeček
  • 37,768
  • 12
  • 121
  • 145
  • 2
    actually, PhantomJS renders the page in memory as far as i know. you can even take screenshots with it! – luksch Jul 25 '13 at 12:55
  • do they also save cookies for login sessions? – Dastan Jan 27 '21 at 20:49
  • 5
    @Pengin The answer is 8 years old now, and a lot has changes. Selenium still rules, PhantomJS no longer exists. And there are new things, like https://github.com/microsoft/Playwright-Java, and many more. I do no longer know which in-memory library is the best. Sorry. – Petr Janeček Jan 27 '21 at 22:28
5

I managed to run Selenium on CentOS server. I have used Chromedriver. As I found the Chromedriver perhaps needs a browser (although the latest Chrome reportedly can run without browser...I didn't try it). But in server you don't need a GUI to run a browser (Google-Chrome) for this purpose.

Following are the steps I have followed:

  • We need Google Chrome to be installed (IF NOT INSTALLED):
# FOR CENTOS
wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm
yum localinstall google-chrome-stable_current_x86_64.rpm

# FOR UBUNTU (I didn't validate it myself, as I only worked with CentOS)
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo dpkg -i google-chrome-stable_current_amd64.deb
sudo apt-get install -f
  • Now check if Google Chrome is installed by checking its version:
google-chrome --version
  • If the above steps are okay, now download Chromedriver for your distribution and according to the Google Chrome's version you have. To get Chromedriver go: Chrome Driver | Downloads
  • Now you have to include several options to run it. (without --no-sandbox the Chrome browser won't run. Some also say to keep this option as the first option. Some other say only this option is enough....read the comments of the post mentioned in the third reference.):
from selenium.webdriver.chrome.options import Options
from selenium import webdriver

options = Options()
options.add_argument('--no-sandbox')
options.add_argument("--headless")
options.add_argument('--disable-dev-shm-usage')
  • Now show the Chromedriver's directory (the following shows for the current directory) and also pass these options:
driver = webdriver.Chrome('./chromedriver', options = options)
  • Finally start Selenium's crawling:
driver.get(YOUR_DESIRED_LINK)

Reference:

hafiz031
  • 2,236
  • 3
  • 26
  • 48
1

Using Selenium, WebDriver, and ChromeDriver

Right now, Selenium opens a full instance of Chrome. In other words, it's an automated solution but not completely headless. However, Selenium can be configured to run headless Chrome with a little work. I recommend Running Selenium with Headless Chrome if you want the full instructions on how to set things up yourself, but I've dropped in some examples below to get you started.

iMath
  • 2,326
  • 2
  • 43
  • 75