9

I want to run a Python script using Selenium WebDriver with PhantomJS as a headless browser on my Raspberry Pi running Raspbian.

I originally wrote the script in OS X where it works fine. But in trying to make it work on the Raspberry, I'm having problems.

When trying to run the script, I get this error:

raise WebDriverException("Can not connect to the Service %s" % self.path)
selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service /usr/bin/phantomjs

Brief version of the script:

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

user_agent = ("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) " +
    "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36")

dcap = dict(DesiredCapabilities.PHANTOMJS)
dcap["phantomjs.page.settings.userAgent"] = user_agent

serv_args = ["--ignore-ssl-errors=false", "--ssl-protocol=tlsv1", 
    "--disk-cache=false"]

driver = webdriver.PhantomJS(executable_path="/usr/bin/phantomjs", 
    desired_capabilities = dcap, service_arguments = serv_args, port=65000)

I have seen other people have problems resembling mine – with varying solutions – most seem to involve building PhantomJS yourself, or clone and install a Github branch adapted for Raspberry (which now is out-of-sync with the main PhantomJS project).

Questions

  • Does anyone know how to resolve the problem – and indeed what the problem really is about?
  • If the solution involves installing the binaries manually to /usr/local/bin or such, how would I do that? The binaries available on the PhantomJS webpage is for linux-x86 and linux-i686, so I'm assuming they won't work on an Raspberry Pi 2 B ARM Cortex A-7 processor.
  • I have also tried to build PhantomJS myself according to these instructions, but the process froze midway through. The Raspberry also doesn't meet the recommended hardware requirements for building.

Background info

  • I am using Python 2.7.9
  • I have created a virtualenv and installed all the Python modules in it; e.g. pip install selenium, and attempt to run the script here
  • I have installed the latest version of PhantomJS via sudo apt-get install phantomjs
  • I have disabled my ufw firewall while testing
P A N
  • 5,642
  • 15
  • 52
  • 103

3 Answers3

20

Ok I will start with the solution, there is a version compiled for arm here phantomjs-linux-armv6l, on the pi run the following commands:

$ cd /tmp
$ wget https://github.com/aeberhardo/phantomjs-linux-armv6l/archive/master.zip
$ unzip master.zip
$ cd phantomjs-linux-armv6l-master
$ bunzip2 *.bz2 && tar xf *.tar

I added:

sudo cp phantomjs-1.9.0-linux-armv6l/bin/phantomjs  /usr/bin

So phantomjs will be on your path.

pi@raspberrypi ~ $ phantomjs --version
1.9.0

pi@raspberrypi ~ $ phantomjs
phantomjs>

Now we have that done, time to test:

pi@raspberrypi ~ $ cat test.py
#!/usr/bin/python
from selenium import webdriver

driver = webdriver.PhantomJS()
driver.get('http://stackoverflow.com/questions/36314771/how-to-install-phantomjs-for-use-with-python-selenium-on-the-raspberry-pi/36388824#36388824')
a = driver.find_element_by_xpath('//*[@id="question-header"]/h1/a')
print(a.text)
print(driver)
pi@raspberrypi ~ $ python test.py 
How to install PhantomJS for use with Python Selenium on the Raspberry Pi?
<selenium.webdriver.phantomjs.webdriver.WebDriver (session="b184e110-f9c4-11e5-aede-7f5c42f062d7")>

From the faq. Starting with PhantomJS 1.5, it is pure headless and there is no need to run X11/Xvfb anymore..

I tried using xvfb-run and exporting the display, using a shell script in init.d to start xvfb, I got a little further being able to run iceweasel from bash headless no problem but still no cigar when it came to phantomjs and selenium. I think it may just come down to an incompatibility between selenium and the version of phantomjs, regardless having 1.9.0 and real headless browsing is a lot more desirable.

I was in the middle of setting up a toolchain and was going to try to compile myself when I found the link above, for anyone interested in cross compiling, crosstools-ng makes life a lot easier.

I am running an arm6, there is also a compiled version for arm7 using 2.0.0, the dependencies are:

sudo apt-get install flex bison gperf ruby perl libsqlite3-dev libfontconfig1-dev libicu-dev libfreetype6 libssl-dev libpng-dev libjpeg-dev python libX11-dev libxext-dev

The install procedure, I have extracted the binary to dropbox:

wget https://www.dropbox.com/s/epj1rji9d239dco/phantomjs
chmod +x phantomjs
sudo cp phantomjs /usr/bin

The original github link is phantomjs-2.0.0-armv7

Mr-Programs
  • 767
  • 4
  • 20
Padraic Cunningham
  • 176,452
  • 29
  • 245
  • 321
0

Here's how I did it :

For 64-bit system, download phantomjs-1.9.7-linux-x86_64.tar.bz2 (12.6 MB).
For 32-bit system, download phantomjs-1.9.7-linux-i686.tar.bz2 (12.9 MB).

Step 1> $ wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-1.9.7-linux-x86_64.tar.bz2

Step 2> $ tar -xvf phantomjs-1.9.7-linux-x86_64.tar.bz2

Step 3> $ cd phantomjs-1.9.7-linux-x86_64

Step 4> $ cd bin

Step 5> $ sudo cp phantomjs /usr/bin

To check phantomjs installed properly perform the following step : 

Step 6> $ phantomjs -h

To install the dependencies on Red Hat based systems:

Step 7> $ sudo yum install fontconfig freetype libfreetype.so.6 libfontconfig.so.1 libstdc++.so.6
Ahmed Soliman
  • 1,662
  • 1
  • 11
  • 16
0

Download this phantomjs file https://drive.google.com/open?id=1x063Krw6mZkRYW4K238a3EyRdklj5Evj

Replace to needed folder.

Give 777 chmod: chmod 777 phantomjs

And try to use.

For Grafana - phantomjs file must be in a folder: /usr/share/grafana/tools/phantomjs/.

This binary file work to me on Banana pi M3 Debian 9. Architecture: arm armv7l. Kernel 4.20.7-sunxi

user3500968
  • 1