0

I'm building a website right now in which I'm trying to execute Javascript code to obtain specific elements of a totally separate site's web page given its URL. I've figured out how to use Selenium.Webdriver on my laptop in Python to achieve this by:

driver = webdriver.Firefox()
driver.get(url)

price = driver.execute_script("var priceElements = document.getElementById('priceblock_ourprice'); var prices = []; for (var i = 0; i < 3; i++) { prices.push(priceElements.children[i].innerHTML); } var price = prices[1] + '.' + prices[2]; return price;")
driver.close()

where it opens up the Firefox browser, runs the JS and then finds this price value that I want. I know there is also a way to do it which is headless and it won't need to physically open up a browser, but I'm trying to figure this out for the case of doing something similar on the website that I'm building.

Is there some way that I can achieve this result but instead of it running on my personal machine, the code runs on the Apache web server? I'm just not sure if this is possible without the machine its running on having a browser that it can open.

I hope I'm clear enough on what I'm doing, but if anyone needs clarification then I'll be happy to answer any questions you may have about this situation.

Blake
  • 35
  • 2
  • The usual way to do this would be with a tool like [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) to scrape the values from the remote webpage. – Matt Morgan Apr 05 '18 at 00:02
  • @MattMorgan I've heard of that and will give it a shot, thanks! – Blake Apr 05 '18 at 02:33

1 Answers1

0

I did something similar with "lxml" and "urllib" Here is a example:

from lxml import html
import urllib

pageLink = ""
page = urllib.urlopen(pageLink)
pageString = page.read()
pageContent = html.fromstring(pageString)
imgLink = pageContent.xpath('//img[@id="img-1"]/@src')[0]
Ricardo
  • 1,308
  • 1
  • 10
  • 21