Downloading PDFs with RSelenium

Question

I'm trying to download multiple pdf's from this website using the "County" level option -- https://public.education.mn.gov/MDEAnalytics/DataTopic.jsp?TOPICID=11 -- using RSelenium. Here's what I have so far, which works for downloading the first instance of the pdf (this output can also be gotten by choosing "County" for level and then clicking the "List files" option on the website):

remDr <- remote_driver$client remDr$open() 
remDr$navigate("https://public.education.mn.gov/MDEAnalytics/DataTopic.jsp?TOPICID=11") # to navigate to a page
frames <- remDr$findElements('css', "iframe")
remDr$switchToFrame(frames[[1]])
remDr$findElement(using = "id",value = "cmbCOLUMN1")$clickElement()
option <- remDr$findElement(using = 'xpath', "//*/option[@value = 'County']")
option$clickElement()
remDr$findElement(using = "id",value = "button1")$clickElement()
report_frame <- remDr$findElement(using = "id",value = "report")
remDr$switchToFrame(report_frame)
remDr$findElement(using = "class", value = "buttonpdf")$clickElement()

How do I use RSelenium to download the rest of the pdfs? I tried to use the $findElements method and tried to use different "using" arguments, but no luck. I'm not very skilled with RSelenium and the prerequisite html/css skills that would help figure this out.

score 0 · Accepted Answer · edited Apr 10 '23 at 06:52

I figured out a solution with the help of https://burtmonroe.github.io/TextAsDataCourse/Tutorials/TADA-RSelenium.nb.html, dropdown boxes in RSelenium, and a stackoverflow user named Russ:

remDr <- remote_driver$client
remDr$open() # to launch
remDr$navigate("https://public.education.mn.gov/MDEAnalytics/DataTopic.jsp?TOPICID=11") # to navigate to a page
frames <- remDr$findElements('css', "iframe")
remDr$switchToFrame(frames[[1]])
remDr$findElement(using = "id",value = "cmbCOLUMN1")$clickElement()
option <- remDr$findElement(using = 'xpath', "//*/option[@value = 'County']")
option$clickElement()
remDr$findElement(using = "id",value = "button1")$clickElement()
report_frame <- remDr$findElement(using = "id",value = "report")
remDr$switchToFrame(report_frame)
Details_El <- remDr$findElements(using = "class", value = "buttonpdf")
# length(Details_El) to get the 339 length
for (i in 1:339) {
  Details_El[[i]]$clickElement()
}

Downloading PDFs with RSelenium

1 Answers1