1

I am trying to write a a single RScript that will scrape college ranking information from https://www.usnews.com/best-colleges/rankings/regional-universities-north. This site requires scrolling down to load more college information.

My hope is to do this without the use of docker or an outside program that requires command line prompts. From my attempts, it seems that RSelenium is defunct as well.

indiemathematician
  • 19
  • 2

1 Answers1

1

Here is a partial answer,

library(RSelenium) 
driver <- rsDriver(browser=c("firefox"))
remDr <- driver$client

remDr$navigate("https://www.usnews.com/best-colleges/rankings/regional-universities-north")
# Name of the college 
remDr$findElement(using = "xpath",'//*[@id="school-0"]/div/div[1]/div[1]/div[1]/h3') -> collegename
collegename$getElementText()
#Rank of the college 
remDr$findElement(using = "xpath",'//*[@id="school-0"]/div/div[1]/div[1]/div[2]/ul/li/a/div') -> rankcollege
rankcollege$getElementText()

This gives you college name and rank.

But to get rank for all colleges you have to loop. To scroll down the page refer, Scrolling page in RSelenium

Nad Pat
  • 3,129
  • 3
  • 10
  • 20