Python Selenium On Local HTML String

Question

I am trying to run Selenium on a local HTML string but can't seem to find any documentation on how to do so. I retrieve HTML source from an e-mail API, so Selenium won't be able to parse it directly. Is there anyway to alter the following so that it would read the HTML string below:

Python Code for remote access:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()
driver.get("http://www.python.org")
assert "Python" in driver.title
elem = driver.find_element_by_class_name("q")

Local HTML Code:

s = "<body>
        <p>This is a test</p>
        <p class="q">This is a second test</p>
     </body>"

should that not be .find_element_by_class_name ? is that the underlying problem or is this just a typo — smoggers, Dec 29 '15 at 20:38

score 21 · Answer 1 · answered Sep 25 '18 at 12:35

21

If you don't want to create a file or load a URL before being able to replace the content of the page, you can always leverage the Data URLs feature, which supports HTML, CSS and JavaScript:

from selenium import webdriver

driver = webdriver.Chrome()
html_content = """
<html>
     <head></head>
     <body>
         <div>
             Hello World =)
         </div>
     </body>
</html>
"""

driver.get("data:text/html;charset=utf-8,{html_content}".format(html_content=html_content))

answered Sep 25 '18 at 12:35

jolancornevin

999
1
9
7

This is limited to 65535 characters. – Remy Ticona Apr 25 '20 at 02:26
Yes and no, it actually depends on the browser that you are using. CF https://stackoverflow.com/a/41755526/5061294. – jolancornevin Apr 27 '20 at 08:37

timbre timbre · Accepted Answer · 2020-05-09T18:18:55.163

16

If I understand the question correctly, I can imagine 2 ways to do this:

Save HTML code as file, and load it as url file:///file/location. The problem with that is that location of file and how file is loaded by a browser may differ for various OSs / browsers. But implementation is very simple on the other hand.
Another option is to inject your code onto some page, and then work with it as a regular dynamic HTML. I think this is more reliable, but also more work. This question has a good example.

edited May 09 '20 at 18:18

answered Dec 29 '15 at 22:12

timbre timbre

12,648
10
46
77

I think I'll go with the injection. Thanks. – user2694306 Dec 29 '15 at 22:28
I found below works for me location = "file:///Users/your_username/github/login_automation/landing.html" driver.get(location) print (driver.page_source) – ravindrab Mar 22 '19 at 18:37

score 6 · Answer 3 · answered Jul 19 '16 at 14:50

6

Here was my solution for doing basic generated tests without having to make lots of temporary local files.

import json
from selenium import webdriver
driver = webdriver.PhantomJS()  # or your browser of choice

html = '''<div>Some HTML</div>'''
driver.execute_script("document.write('{}')".format(json.dumps(html)))
# your tests

answered Jul 19 '16 at 14:50

leontrolski

365
3
8

Not sure why this was downvoted, but seems to do the trick for me. – laverick Jan 11 '18 at 22:09
1

Sad, but throws a JavascriptException: SecurityError: The operation is insecure. – wp78de Sep 03 '18 at 04:27
`re.escape` is more appropriate than `json.dumps` to turn the special characters of a string into escaped characters. The latter won't escape single quotes, for example, and this breaks the string within `document.write()` – Mutoh Jan 09 '20 at 19:26

score -1 · Answer 4 · answered Dec 29 '15 at 20:44

-1

If I am reading correctly you are simply trying to get text from an element. If that is the case then the following bit should fit your needs:

    elem = driver.find_element_by_class_name("q").text
    print elem

Assuming "q" is the element you need.

answered Dec 29 '15 at 20:44

Corey

133
2
11

I'm trying to parse a local string. So rather that feed a url to "driver.get", I would want to pass the html string to it. – user2694306 Dec 29 '15 at 21:01

Python Selenium On Local HTML String

4 Answers4