Selenium task becomes slower and slower

Question

not sure what the Title should be, please feel free to edit. I am new to Selenium, and was just playing around with it. I tried to experiment on one of my favorite site(10fastfingers.com) which I frequently use for practicing typing. I tried to automate the typing using the latest versions of both Selenium and Python(3.5.*). The automation works, but is REALLY REALLY slow.

Here is my sample script, I have no idea why it is slowing down. Would be great it someone would help me out here. I was expecting the script to go more than 100 WPM, but it only reaches 37 WPM

Problem is, only two lines of words are visible, so, it does not take all the words at once, The words keep appearing as you keep typing them. So, I could not take in all the words at once, and store it in a list, and then loop over them later.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()
driver.get("http://10fastfingers.com/typing-test/english")
input = driver.find_element_by_id("inputfield")
for i in range(300):
    word = driver.find_element_by_class_name("highlight")
    word = word.text
    print("Typing word ", word)
    input.send_keys(word)
    input.send_keys(Keys.SPACE)

UPDATE

I tried getting all the words before instead of using find_element_* inside the loop as suggested by @alecxec. For this, I had to use, BeautifulSoup to parse the HTML, as it was in String format,and I could not use any of the find_element_* functions on it. The following script improved the Typing Speed to 138 WPM. But, I noticed that the first 20 seconds of typing is REALLY fast, and then the speed starts falling gradually. Please feel free to try out the script on you machine, and let me know the result of the test(WPM) that the script achieved on your system. Maybe it depends on the system configuration as well. I don't know. Is this some memory issue? or Code issue? Any idea how i can fix this.

Modified code

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
driver = webdriver.Firefox()
driver.get("http://10fastfingers.com/typing-test/english")
input = driver.find_element_by_id("inputfield")

script = "return document.getElementById('row1').innerHTML"
all_words= driver.execute_script(script)
soup = BeautifulSoup(all_words, 'html.parser')
words = soup.find_all('span')
for word in words:
    text = word.text
    input.send_keys(text + Keys.SPACE)

UPDATE 2

I restarted my system, and closed all other applications, and the script ran like magic, it typed all 361 available words before the timer even reached 60 secs, and the result was 361 WPM. So, this does depends on the available RAM.

alecxe · Answer 1 · 2016-06-30T20:59:37.223

5

Every .find_element_*, .text, .send_keys() etc calls are WebDriver commands which involve HTTP request/response (see JSON wire protocol) processing. You can dramatically decrease the amount of WebDriver commands you send by getting the word value before the loop:

word = driver.find_element_by_class_name("highlight").text
for i in range(300):
    print("Typing word ", word)
    input.send_keys(word + Keys.SPACE)

I've also joined the send_keys() into a single call.

Note that there is a better way to handle this particular use case. All the words are actually there in the HTML, you just need to get them. .text would only return a visible text only, use get_attribute("textContent") instead:

input_elm = driver.find_element_by_id("inputfield")

words_to_type = [word.get_attribute("textContent") + Keys.SPACE 
                 for word in driver.find_elements_by_css_selector("#row1 > span")]
for word in words_to_type:
    input_elm.send_keys(word)

Or, how about that? Use Action Chains - prepare the actions beforehand and then perform (360 words per minute on my machine):

from selenium.webdriver import ActionChains

input_elm = driver.find_element_by_id("inputfield")

actions = ActionChains(driver)
actions = actions.move_to_element(input_elm).click()
for word in driver.find_elements_by_css_selector("#row1 > span"):
    actions = actions.send_keys(word.get_attribute("textContent") + Keys.SPACE)
actions.perform()

edited Jun 30 '16 at 20:59

answered Jun 30 '16 at 20:23

alecxe

462,703
120
1,088
1,195

Problem is, only two lines of words are visible, so, it does not take all the words at once, The words keep appearing as you keep typing them. So, I had to take this approach. If you could just open the site, and try out the test once. – Parthapratim Neog Jun 30 '16 at 20:24
@ParthapratimNeog yeah, gotcha, let me improve that with an alternative option. – alecxe Jun 30 '16 at 20:32
I tried [this](http://stackoverflow.com/questions/13047056/how-to-read-text-from-hidden-element-with-selenium-webdriver) out, to get all the words, but the js execution returns a string, so I am not able to use the `find_elements_*` function on the response. – Parthapratim Neog Jun 30 '16 at 20:37
Please check my Update on the question. – Parthapratim Neog Jun 30 '16 at 20:57
@ParthapratimNeog updated the answer with an option. What results do you get with it? Thanks! – alecxe Jun 30 '16 at 21:00
@ParthapratimNeog and, there is one more insane option posted: using Action Chains. Check it out. – alecxe Jun 30 '16 at 21:04
I would try all your suggestions out, and let you know in comments tomorrow. Thanks for the help. Do check my second update. – Parthapratim Neog Jun 30 '16 at 21:10

score 1 · Answer 2 · answered Jun 30 '16 at 23:27

I love stuff like this. I did mine in Java first like this.

while (true)
{
    String word = driver.findElement(By.cssSelector("span.highlight")).getText();
    driver.findElement(By.id("inputfield")).sendKeys(word + " ");
}

This got me 361 wpm but I noticed that I ran out of words about halfway through. My second attempt was much faster but it was too fast. All the words were wrong.

List<WebElement> words = driver.findElements(By.cssSelector("span[wordnr]"));
WebElement input = driver.findElement(By.id("inputfield"));
System.out.println(words.size());
for (WebElement word : words)
{
    input.sendKeys(word + " ");
}

I realized that my first attempt was using highlight which meant that it would only go as fast as the page registered the word being successfully entered (thus triggering the highlight class change). I think ~360 is as fast as you can get because of the slowness (relatively) of the page.

Cool, i think we had similar approach at first. Can you tell me of some other cool things that we can do with Selenium? — Parthapratim Neog, Jul 01 '16 at 04:40
There have been many books written on what you can do with Selenium... and they aren't comprehensive. I'm not sure what you are asking. :) — JeffC, Jul 01 '16 at 23:28

Selenium task becomes slower and slower

2 Answers2