can't execute rsDriver (connection refused)

Question

I can't get anywhere with R selenium. Here's the first step and my output:

library(RSelenium)
rD <- rsDriver()
# checking Selenium Server versions:
#   BEGIN: PREDOWNLOAD
# BEGIN: DOWNLOAD
# BEGIN: POSTDOWNLOAD
# checking chromedriver versions:
#   BEGIN: PREDOWNLOAD
# BEGIN: DOWNLOAD
# BEGIN: POSTDOWNLOAD
# checking geckodriver versions:
#   BEGIN: PREDOWNLOAD
# BEGIN: DOWNLOAD
# BEGIN: POSTDOWNLOAD
# checking phantomjs versions:
#   BEGIN: PREDOWNLOAD
# BEGIN: DOWNLOAD
# BEGIN: POSTDOWNLOAD
# [1] "Connecting to remote server"
# Error in checkError(res) : 
#   Undefined error in httr call. httr output: Failed to connect to localhost port 4567: Connection refused
# In addition: Warning message:
#   In rsDriver() : Could not determine server status.

What did I miss ?

moodymudskipper · Accepted Answer · 2020-09-04T21:17:34.360

39

Note: this answer is meant for Windows

When trying to run the deprecated checkForServer() Selenium offers two options:

use rsDriver
use Docker

see:

RSelenium::checkForServer()
# Error: checkForServer is now defunct. Users in future can find the function in 
# file.path(find.package("RSelenium"), "examples/serverUtils"). The
# recommended way to run a selenium server is via Docker. Alternatively
# see the RSelenium::rsDriver function.

Everybody seems to have issues with rsDriver and Docker is the recommended option so we'll go this route:

install docker
run it, restart computer as requested
pull image by running in command line: docker pull selenium/standalone-firefox(or chrome instead of firefox) or in R shell('docker pull selenium/standalone-firefox')
start server by running in command line: docker run -d -p 4445:4444 selenium/standalone-firefox or in R shell('docker run -d -p 4445:4444 selenium/standalone-firefox')
Then run remDr <- remoteDriver(remoteServerAddr = "localhost", port = 4445L, browserName = "firefox'") . The doc suggests something different with a virtual machine but i couldn't get it to work.

With this I was set, here is my code:

shell('docker run -d -p 4445:4444 selenium/standalone-firefox')
remDr <- remoteDriver(remoteServerAddr = "localhost", port = 4445L, browserName = "firefox")
remDr$open()
remDr$navigate("http://www.google.com/ncr")
remDr$getTitle()
# [[1]]
# [1] "Google"

The doc for more info:

edited Sep 04 '20 at 21:17

answered Jul 30 '17 at 15:13

moodymudskipper

46,417
11
121
167

1

Any similiar tutorial how to start using RSelenium on Mac? – MLEN Jul 31 '17 at 18:08
I can't help you much unfortunately, see maybe this : https://stackoverflow.com/questions/35619490/how-can-i-start-rselenium-in-mac or this https://stackoverflow.com/questions/31948860/rselenium-in-mac – moodymudskipper Jul 31 '17 at 20:06
You've used the same command for pulling the image and starting the server: `docker run -d -p 4445:4444 selenium/standalone-firefox` Shouldn't the first be some invocation of `docker pull` instead? (using the same command for both tasks works, though, because `docker run` pulls the image if it is not available locally). – Frank Schmitt Aug 02 '17 at 10:45
You're right Frank, I had made a copy and paste mistake, it's fixed, thanks. I also harmonized the use of `firefox` and `chrome` as they was some ambiguity – moodymudskipper Aug 02 '17 at 11:36
also it seems you have one `'` too much in `browserName = "firefox'"` – Triamus Oct 18 '17 at 21:06
don't you mean: `system('docker run -d -p 4445:4444 selenium/standalone-firefox')` ; instead of `shell` – deann Mar 04 '18 at 11:53
it will have the same effect, `shell is a more user-friendly wrapper for system` from `?shell` – moodymudskipper Mar 04 '18 at 12:00
1

does this mean you are running RSelenium from inside Docker? – QHarr Jul 28 '19 at 02:02
3

when I run this I get the following error ``Undefined error in httr call. httr output: Failed to connect to localhost port 4445: Connection refused`` – Nathan123 Sep 16 '19 at 16:47
in R: system() is for mac, shell() is for Linux – Adam Bricknell Sep 01 '21 at 20:08

score 20 · Answer 2 · edited Jan 25 '23 at 11:01

After using RSelenium/rsDriver for a while without issue, I recently got a very similar error message as OP, but couldn't use Docker to solve it (mostly for organizational reasons). Here's how I solved it:

Taking a look at the function documentation, it turns out that rsDriver() is default configured to launch the Selenium server with the latest versions of both Chrome and Firefox (Gecko) drivers, even if you have specified the other browser to run. In my case, I was specifying browser = "firefox", but a new Chrome driver had auto-downloaded on the most recent run and, for whatever reason, wasn't playing nice with Selenium. That was the root cause of the issue.

It's not particularly intuitive, and the "Connection refused" message doesn't really help, but following the error advice and checking the server log made me realize that Selenium was still trying to load Chrome drivers, even though I was using Firefox:

rd <- rsDriver(browser = "firefox")
# Yields an error message after doing pre/post downloads:

#> Could not open firefox browser.
#> Client error message:
#> Undefined error in httr call. httr output: Failed to connect to localhost port 4567: Connection refused
#> Check server log for further details.
#> Warning message:
#> In rsDriver(browser = "firefox" :
#>  Could not determine server status.

rd$server$log()

> $stderr
> [1] "Error: Could not find or load main class c(-Dwebdriver.chrome.driver=\"C:\filepath\to\chromedrivers\109.0.5414.25.chromedriver.exe\","

> $stdout
> character(0)

Anyway, if you specify rsDriver() to run with either an older driver version, or just pass in NULL so it doesn't look for a driver at all, it might fix the issue (it worked for me).

# Default parameters of the rsDriver() function:

rsDriver(port = 4567L,
         browser = c("chrome", "firefox", "phantomjs", "internet explorer"),
         version = "latest",
         chromever = "latest",
         geckover = "latest",
         iedrver = NULL,
         phantomver = "2.1.1",
         verbose = TRUE,
         check = TRUE,
         ...)

# Specify an older driver version you know works (or just use NULL), 
# even if it doesn't correspond to the browser you're using:

rd <- rsDriver(browser = "firefox",
               chromever = "109.0.5414.25") # alt: chromever = NULL

In case you're looking for the filepath that these drivers get saved to when auto-downloaded by RSelenium (e.g. maybe you want to find the name of an older driver version), on Windows, it should be ~/AppData/Local/binman.

Shoutout to the responses on this question, which also helped point me in the right direction!

Fantastic answer! I was worried I was going to have to put in a request to get docker installed (my company is fairly locked down permission wise). Your solution was a lifesaver! Can't believe it is checking for chrome when I am using firefox XD. — JRP, Dec 09 '22 at 18:27
Thank you so much for this answer! I was going crazy with the issue and could not find a solution. — U_jex, Dec 13 '22 at 15:03
On second thought, it worked for me once, but I can't reproduce the fix. I'm now continuing to get the same error message. Frustrating. — Christopher Carroll Smith, Dec 15 '22 at 18:19
Things that worked for others: 1) deleting the LICENSE.chromedriver file and 2) rolling back wdman from version 0.2.6 to 0.2.5. What finally worked for me, though, was updating all of the packages that wdman is dependent upon. It appears that I had some pretty old versions of packages like yaml and jsonlite that hadn't been updated in a long time. Once I updated all of these, rsDriver started working again. — Christopher Carroll Smith, Dec 16 '22 at 16:23
thanks for figuring this out. all i had to change was adding `chromever = NULL` — hypothesis, Dec 31 '22 at 17:06
I tried everything I could to get chrome to work, spent hours, and nothing did. Then switching to Firefox and using chromever = NULL worked. Like. A. Charm. Thank you! — James Cutler, May 16 '23 at 19:11
Same, setting up NULL chromever and firefox browser worked for me — Alberto Agudo Dominguez, Jul 17 '23 at 12:50

score 4 · Answer 3 · answered Jan 05 '19 at 18:50

In case that is still useful, I was running into the same problem today and was able to fix it by installing a Java Development Kit (Java SE Development Kit 11.0.1).

I was getting an error message from my computer to that effect, as well as the same R error as mentioned in this question, and it fixed it.

For a reproducible example, I was able to replicate this tutorial.

sessionInfo() R version 3.4.4 (2018-03-15) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS High Sierra 10.13.5

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] fr_CA.UTF-8/fr_CA.UTF-8/fr_CA.UTF-8/C/fr_CA.UTF-8/fr_CA.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RSelenium_1.7.5

loaded via a namespace (and not attached):
[1] Rcpp_0.12.17     XML_3.98-1.11    binman_0.1.1     assertthat_0.2.0 rappdirs_0.3.1   bitops_1.0-6    
[7] R6_2.2.2         jsonlite_1.5     semver_0.2.0     httr_1.3.1       curl_3.2         xml2_1.2.0      
[13] subprocess_0.8.3 tools_3.4.4      wdman_0.2.4      yaml_2.1.18      compiler_3.4.4   caTools_1.17.1  
[19] openssl_1.0.1

The site linked to is not working - could you add details of the fix in your answer? — msm1089, Apr 29 '23 at 09:17
I'm afraid the website is down, and there doesn't seem to be a copy of it on the wayback machine :/ — pollytatouin, Apr 30 '23 at 22:14
Ah shame! An alternative tutorial is at https://joshuamccrain.com/tutorials/web_scraping_R_selenium.html (not that it will help until the issue is fixed). For me I could resolve the issue by deleting the license files from the chromedriver downloads. — msm1089, May 01 '23 at 09:58

score 3 · Answer 4 · edited Jul 20 '23 at 21:50

In 2023 I'm using chrome and getting this error and the solution that worked for me was to delete LICENSE.chromedriver in the following directory (on Windows 10):

C:\Users\YOURUSERNAME\AppData\Local\binman\binman_chromedriver\win32\114.0.5735.16

or

C:\Users\YOURUSERNAME\AppData\Local\binman\binman_chromedriver\win32\YOURCHROMEVERSION

I don't know why it works, it was suggested by a YouTuber who's name I can't remember and it worked for on three separate installs.

score 2 · Answer 5 · edited Apr 19 '23 at 21:08

2

Per the Cran vignette I had to run:

shell("docker-machine ip")

Then set the remoteServerAddr to the IP which was returned.

edited Apr 19 '23 at 21:08

M--

25,431
8
61
93

answered Feb 18 '20 at 18:54

Quinn

21
1

can't execute rsDriver (connection refused)

5 Answers5

Linked