1

I am interested in the stats in the English Premier League. So, I try to get data from this official site https://www.premierleague.com/stats/top/players/total_pass

I am using R and RSelenium package. 

library(rvest)
library(httr)
library(RSelenium)

remDr <- remoteDriver(port = 4445L)
remDr$open()
remDr$navigate('https://www.premierleague.com/stats/top/players/total_pass')
getsource <-remDr$getPageSource()
name<- read_html(getsource[[1]]) %>% html_nodes("strong") %>% html_text()

But I got the some problems. There are some categories of data such as seasons, positions, clubs and so on.

So, I thought that I can get data based on these categories. But I did not know how to select specific things in the dropdown box using Rselenium in this site.

I thought that filenElement and clickElement are useful functions for this. However, I do not know how I should handle these functions to select specific conditions such as 2016/17 season and Goalkeeper position.

Please give me an advice for this.

kmangyo
  • 1,147
  • 1
  • 12
  • 13

1 Answers1

6

Using the following code I was able to get the browser to select the 2014/15 season. You will need to inspect the contents of the various drop-downs and expand this as required.

rD <- rsDriver(port=4444L,browser="chrome")
remDr <- rD$client

#navigate to main page
remDr$navigate('https://www.premierleague.com/stats/top/players/total_pass')

#find 'filter by season' box and click it
webElem <- remDr$findElement(using = 'xpath', value = "//*[@data-dropdown-block='FOOTBALL_COMPSEASON']")
webElem$clickElement()

#find 2014/15 season and click it
webElem1 <- remDr$findElement(using = 'xpath', value = "//*[@data-option-name='2014/15']")
webElem1$clickElement()
Andrew Gustar
  • 17,295
  • 1
  • 22
  • 32
  • Thank you for your comment. But I have got this error message. Error: Summary: ElementNotVisible Detail: An element command could not be completed because the element is not visible on the page. class: org.openqa.selenium.ElementNotVisibleException Further Details: run errorDetails method What is wrong with my code? – kmangyo Apr 10 '17 at 15:06
  • In that case you need to get it to click the dropdown box first - I have amended the code above to illustrate this. You can then `getPageSource` as before and extract the data you need. You could also do this after clicking the dropdown box in order to extract the list of options, if, for example, you wanted to use them to build a loop. – Andrew Gustar Apr 10 '17 at 16:28