How to find an element on a webpage using Selenium WebDriver?

Question

I trying to automate a task using Selenium WebDriver for FireFox on YouTube.

I have a playlist created on YouTube and I want to get the hyperlink of all the videos in that playlist.

The html looks like this:

<a href="/watch?v=StJLvbPIvTw&amp;list=PLt5xbw4ekDQssXxfaIfh_XbKe-iuOTZo_&amp;index=1" title="ADELE - Skyfall (Official video HD)" class="yt-uix-tile-link yt-uix-sessionlink" data-sessionlink="feature=plpp_video&amp;ei=RodgUazfOKWlhAHOioGoDA">
    <span class="title video-title" dir="ltr">ADELE - Skyfall (Official video HD)</span>
  </a>

I tried finding the element using the find_by_partial_link_text but failed.

My ultimate motive is to get the hyperlink for all the videos in the playlist so that i can pass it to next script to access them individually. Any help will be highly appreciated.

Note: I found an answer in SO similar to this SO but the answer provided here is iterating over a WebElement object which throws an exception in my case saying object is not iterable.

EDIT:

def init():

    d = webdriver.Firefox()
    d.implicitly_wait(15)
    print "in init"
    return d

def youtube(d, uname, pwd):

    link_list = []
    d.get("http://www.youtube.com")
    print "in you"
    signin = d.find_element_by_partial_link_text("Sign in")
    signin.click()

    email = d.find_element_by_id("Email")
    passwo = d.find_element_by_id("Passwd")
    submit = d.find_element_by_id("signIn")

    email.send_keys(uname)
    passwo.send_keys(pwd)
    submit.click()

    list = d.find_element_by_partial_link_text("Playlists")
    list.click()

    play = d.find_element_by_partial_link_text("Fav songs")
    play.click()
    print play

    link_list = d.find_element_by_xpath('//*[@id="playlist-pane-container"]/div[1]')
    print "done"
    print link_list # prints None here
    hr = link_list.get_attribute("css=a@href")
    print hr

@RossPatterson added the code I have tried. Any inputs will be immensely appreciated. — abhi, Apr 08 '13 at 19:55
You have 3 uses of `find_element_by_partial_link_text()`. None of them come anywhere near to matching the HTML fragment you showed. — Ross Patterson, Apr 08 '13 at 23:06
@RossPatterson I was first trying to do that with 'find_element_by_partial_text()' but it was not helping, so instead i tried using the xpath to find all the elements like that on the page as suggested on this SO question http://stackoverflow.com/questions/8121886/accessing-selenium-web-elements-with-python but instead i getting an empty list. — abhi, Apr 09 '13 at 06:34

e1che · Accepted Answer · 2013-04-08T08:54:14.713

Try to get link by : String hrefSpecs = driver.findelement(By.cssSelector("a.yt-uix-tile-link.yt-uix-sessionlink")).getAttribute("href")

normally you get

/watch?v=StJLvbPIvTw&list=PLt5xbw4ekDQssXxfaIfh_XbKe-iuOTZo_&index=1

so you substring that like :

String sublink = hrefSpecs.substring(0,20); // check the 20 i'm not sure

and you'll get

/watch?v=StJLvbPIvTw

since you get this you can write something like that

String youtube = "www.youtube.com";
String link = youtube + sublink;

and you finally get : www.youtube.com/watch?v=StJLvbPIvTw

But if you have a page with all links, you'll build something like :

List<String> listLink = driver.findElements(By.cssSelector("a.yt-uix-tile-link.yt-uix-sessionlink")).getAttribute("href");

and you 'll iterate that with a foreach with previous substring etc. i let you test and tell me if you have problems.

EDIT : WebElement is not iterable, List<WebElement> or [] in your case are iterable. I wrote it in Java, i hope you can "translate" it =)

score -1 · Answer 2 · answered Apr 07 '13 at 03:19

-1

you can get the link by simple string manipulation.

load the string in a variable, split the string by the spaces, the second element will give you the href + link, remove the href part and you will get the link.

try this

html = '<a href="/watch?v=StJLvbPIvTw&list=PLt5xbw4ekDQssXxfaIfh_XbKe-iuOTZo_&index=1" title="ADELE - Skyfall (Official video HD)" class="yt-uix-tile-link yt-uix-sessionlink" data-sessionlink="feature=plpp_video&ei=RodgUazfOKWlhAHOioGoDA"> <span class="title video-title" dir="ltr">ADELE - Skyfall (Official video HD)</span> </a>'

parts = html.split(" ")

link = parts[1][6:-1]

answered Apr 07 '13 at 03:19

scottydelta

1,786
4
19
39

Thanks for the answer.But I have 100+ videos in the playlist and i think it will not be efficient to manually copy the html. I am trying to automate the task – abhi Apr 07 '13 at 08:18
I thought you had separate file for each video, anyways you can always use string manipulation to extract all the videos' html from the html source of the page. – scottydelta Apr 07 '13 at 14:31
Oh, the horror! This is Selenium code, the API has plenty of methods to make this sort of thing much easier. Given an element `blah`, something as simple as `for element in driver.find_elements_by_tagname("a"): print element.get_attribute("href");` would do the job. – Ross Patterson Apr 08 '13 at 23:08

How to find an element on a webpage using Selenium WebDriver?

2 Answers2