0

Based off this question, the OP wants to scrape the table "All Holdings," from this page - scroll down to the yellow part. The table shows the first 10 rows, but can expand to quite a few more.

Both of my rvest and RSelenium solutions only take the first 10 rows, when we want the entire table. My code:

rvest code

library(tidyverse)
library(rvest)

etf_url <- "http://innovatoretfs.com/etf/?ticker=ffty"

etf_table <- etf_url %>%
  read_html %>%
  html_table(fill = T) %>% 
  .[[5]]

RSelenium code

library(RSelenium)
library(rvest)

remDr <- remoteDriver(port = 4445L, remoteServerAddr = "localhost",
                  browserName = "chrome")
remDr$open()
remDr$navigate("http://innovatoretfs.com/etf/?ticker=ffty")
page <- read_html(remDr$getPageSource()[[1]])
table <- html_table(page, fill = TRUE, header = T)
table[[5]]

How can we get the FULL table? Thanks.

papelr
  • 468
  • 1
  • 11
  • 42

1 Answers1

1

Following should expand the table - didn't test it in Selenium but it should work.

remDr$executeScript("__doPostBack('ctl00$BodyPlaceHolder$ViewHoldingsLinkButton','')", args = list())
UpsideDownRide
  • 528
  • 1
  • 4
  • 12
  • Awesome - where should that placed in the script above? Under the `remDr$navigate` argument? – papelr Jul 16 '18 at 01:45
  • It returned: `Selenium message:unknown error: 'args' must be a list (Session info: chrome=67.0.3396.99) (Driver info: chromedriver=2.40.565383 (76257d1ab79276b2d53ee976b2c3e3b9f335cde7),platform=Linux 4.9.87-linuxkit-aufs x86_64) Error: Summary: UnknownError Detail: An unknown server-side error occurred while processing the command. Further Details: run errorDetails method` – papelr Jul 16 '18 at 01:45
  • 1
    I have updated my answer - args were missing from executeScript call. Check if that resolves the issue. Unfortunately I cannot check it myself as installing RSelenium is a pain and I don't have time to fight with it currently. – UpsideDownRide Jul 16 '18 at 03:16
  • Still returns the same error message, unfortunately. Oh well – papelr Jul 16 '18 at 13:21