0

First question asked here.

I need to scrape the url generated by a JS onclick function of a website. So, I think Selenium could do the job, but I can't figure out how to code that without opening other tabs/windows and don't even know if it's possible.

Here's the onclick JS function:

'tr class="linhares" onclick="javascript:grv(0,1,1);">

So a basically need the url returned by this function.

Hope you can help me, thanks!

leo04
  • 45
  • 7
  • It seems a little weird that the function grv() just returns a URL. It doesn't print it anywhere, redirects the browser, or anything like that? – user1337 Jun 24 '16 at 14:19
  • In fact, it opens a new tab with the url dynamically generated. It's like a hidden destination opening. – leo04 Jun 24 '16 at 14:27
  • Will the function grv() always be the same, or are you trying to solve the general case of scraping any such function? – user1337 Jun 24 '16 at 14:36
  • It's always the same, it's really specific for my case. – leo04 Jun 24 '16 at 14:37
  • In that case, can you post the code for grv()? – user1337 Jun 24 '16 at 14:40
  • If I had the function, I wouldn't even need to scrape its result. I'm trying to scrape it's result from a webpage, don't have its code. – leo04 Jun 24 '16 at 15:00

1 Answers1

0

See an example about switching to new window and switch back below:

driver.get("your-application-URL")
time.sleep(3)
window_before = driver.window_handles[0]
windowHandlesAllBefore = driver.window_handles
linkElements = driver.find_elements_by_css_selector("[onclick='javascript:grv(']")

for aLink in linkElements:
    aLink.click()
    time.sleep(3)
    newWindowHandle = list(set(driver.window_handles) - set(windowHandlesAllBefore))[0]
    print(newWindowHandle)
    driver.switch_to.window(newWindowHandle)
    print(driver.current_url)
    driver.close()
    driver.switch_to.window(window_before)
Buaban
  • 5,029
  • 1
  • 17
  • 33
  • Thanks for sharing! As I just need the url of the new window/tab, is there any way to have the browser working with no style (no pictures, no heavy elements) to make it load as fast as possible. – leo04 Jun 25 '16 at 19:33
  • @leo04 You just need to change webdriver to PhantomJS. See example [here](http://stackoverflow.com/questions/13287490/is-there-a-way-to-use-phantomjs-in-python) – Buaban Jun 26 '16 at 01:52
  • Thanks again, Buaban. I'm having issues with the fifth line og your code. InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified – leo04 Jun 27 '16 at 18:19
  • Thanks again! In this time I have built a solution with xpath selector – leo04 Jun 28 '16 at 17:53
  • @leo04 your welcome. If this answer solve your problem, please mark it as the correct answer. – Buaban Jun 29 '16 at 00:46