how to find element in a webpage using Rselenium?

Question

I have the following tag in a webpage:

<a target="PARENT" href="/bin-din/WebOb/mom.ko/6/wo/asaksdaksjd
/5.1.5.5.33.23.23">View Data Set</a>

How can I lookup this element in Rselenium? For example if my current session is saved in remDr, what should I search for:

webElem <- remDr$findElement(??)

I need to search the element using it's display link (View Data Set) since the href link changes over time. Thanks much for your help

have you looked into the XML library? It has extensive capabilities to parse and search through the nodes of a webpage. I'd especially suggest looking into the xpath functions in the XML library. — miles2know, Apr 24 '15 at 18:47
@miles2know great thanks, I already tried some of them it just didn't work. Thank you though — mamal, Apr 24 '15 at 20:19

Pierre L · Answer 1 · 2015-04-24T19:31:24.433

3

Try

library(XML)
fileUrl <- ("http:\\wherever you got your file")
doc <- htmlTreeParse(fileUrl, useInternal=T)
xpathSApply(doc, "//a[@href]", xmlGetAttr, "href")

Demonstration:

fileUrl <- "http://kimkardashianonline.org/"
doc <- htmlTreeParse(fileUrl, useInternal=T)
xpathSApply(doc, "//a[@href]", xmlGetAttr, "href")
[1] "http://kimkardashianonline.org/?page_id=2"                                        
[2] "http://www.kimkardashianonline.org/gallery/"                                      
[3] "http://www.kimkardashianonline.org/icons/"                                        
[4] "http://#"                                                                         
[5] "http://kimkardashianonline.org/?page_id=42"

edited Apr 24 '15 at 19:31

answered Apr 24 '15 at 19:15

Pierre L

28,203
6
47
69

thanks for your response. It works great if the links look like but how about if the html tags look like this: , what should I do in this case? How should I change your code to detect the links for these tags? Thanks a lot – mamal Apr 25 '15 at 01:46
Look at this link [Intro to XML](http://www.omegahat.org/RSXML/shortIntro.pdf). The second page has the code that we used and it has more ideas to replace `xmlGetAttr`. Like xmlName , xmlAttrs , xmlChildren and xmlValue – Pierre L Apr 25 '15 at 13:43
warning: my R Studio crashed when I tried `xmlChildren`. – Pierre L Apr 25 '15 at 13:46

score 2 · Answer 2 · answered Apr 25 '15 at 14:20

2

In the answer by @plafort the [@href] was not needed unless you knew ahead of time what you wanted to set the href attribute value to. So here is a general way forward. This works for this url request. Obviously '_blank' isn't what you'd want.


    library(XML)
    library(RCurl)
    gSite <- getURL("http://www.sitepoint.com/web-foundations/target-html-attribute/") 
    sParse <- htmlParse(gSite)
    xpathSApply(sParse, "//a[@target='_blank']", xmlGetAttr, "href")

answered Apr 25 '15 at 14:20

miles2know

737
8
17

Thanks so much its working!!!! The only issue is that it's only showing the first tag in my html webpage, however, I have five more of these links in that page, how should I find a list containing all of them? Thanks much – mamal Apr 27 '15 at 18:38
humph...could you post a quick example of the tags you want to grab. The code above gets all of the hrefs for – miles2know Apr 27 '15 at 18:41
Thanks, you are right. your code is working on any html. Something is wrong on my side, I will try to figure it out myself, otherwise I will get back to you again. Thanks – mamal Apr 27 '15 at 21:16
Thanks, this solution solved my problem as well. :) – Diego Rodrigues Jun 11 '21 at 20:14

score 2 · Answer 3 · answered Feb 16 '18 at 20:22

2

You can use : webElem$getElementAttribute("href")

answered Feb 16 '18 at 20:22

John Smith

1,604
4
18
45

how to find element in a webpage using Rselenium?

3 Answers3

Linked