How to remove CSS loading and Selenium text

Question

How to make my Selenium script faster, guys. I have this Selenium script:

import time                                     
from selenium import webdriver                
from selenium.webdriver.support.wait import WebDriverWait                                       
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support import expected_conditions as EC                                
from selenium.webdriver.common.by 
import By from selenium.webdriver.common.keys import Keys 
from xvfbwrapper import Xvfb                                                                                                                    
display = Xvfb()                                
display.start()                                                                                 
option = Options()                              
prefs = {'profile.default_content_setting_values': {'images': 2}}                                
option.add_experimental_option('prefs', prefs) 
driver = webdriver.Chrome(options = option)      
driver.get("https://www.nike.com.br/chuteira-nike-premier-2-sala-unissex-153-169-171-309321")
wait = WebDriverWait(driver, 5)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.cc-allow'))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//label[@for='tamanho__idM40F395']"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button#anchor-acessar-unite-oauth2'))).click()

Guys, the problem is that this script is slow. How can I wait only for the time in this scenario instead of waiting longer than the time in this scenario to click on an element? Because I think that because of this excerpt wait = WebDriverWait (driver, 5) the page expects more than it should to click on the elements, so I just wanted to wait for the time in this scenario, and also notice that in this script I took the load of images, I did this to optimize the script, is it possible to remove the loading of images and text? I would like to remove some html and css text elements to load faster. Can someone tell me how can I do this?

What are you trying to target with this selector `"//label[@for='tamanho__id40']"`? I looked at all the label elements on the page and none of them have a `for` attribute like that. — C. Peck, May 24 '21 at 20:30
also, youve tagged both python-3.x and python-2.7 for your question. Which version are you using? — C. Peck, May 24 '21 at 20:31
@C.Peck there are dozens of label elements with the `for` attribute attached to them on the page, but none of them have `tamanho__id40` as a value — JD2775, May 24 '21 at 20:58
@JD2775, yes that is more clearly what I saw too. (or anything even resembling `tamanho__id40` for that value) — C. Peck, May 24 '21 at 20:59
@JD2775 I edited the question had put the URL and the wrong is right now :) — , May 24 '21 at 21:10
OK, thank you for posting your working code. However, I'm still not sure what you want to do. You say, "how can I wait only for the time in this scenario instead of waiting longer than the time in this scenario to click on an element?" What are you meaning by "*only the time in this scenario*"? — C. Peck, May 24 '21 at 21:15
In particular, I know you are looking for " i would like to remove some html and css text elements to load faster", but I don't think there is a way to do this with selenium that would actually speed things up. So I'm trying to understand, big picture, what is your problem with your current code? — C. Peck, May 24 '21 at 21:19
^ what @C.Peck said.....also, `wait = WebDriverWait(driver,5)` does not mean it will wait 5 seconds until it clicks on the element. I wanted to make sure you weren't confused by that as it appears you may be. If the element is found in the DOM it will click it immediately. — JD2775, May 24 '21 at 21:30
I suppose you could try this if you really wanted to disable everything https://stackoverflow.com/questions/49031428/how-to-disable-css-in-python-selenium-using-chromeoptions — JD2775, May 24 '21 at 21:33
When I said in this scenario you mean necessary, I wrote it wrong, as to the purpose of the script, is that I really want to create a script in which I can add the shoes I want in the cart, but I want the script to be quick, I don't know how to use it Selenium was the best option but this is why I wanted to make it quick @JD2775 — , May 24 '21 at 21:42
@JD2775 The problem with this link is that it does not disable css, and I want to disable as much unnecessary stuff as possible — , May 24 '21 at 21:44

vitaliis · Answer 1 · 2021-05-25T04:51:16.100

I've made some research and below are my results. You have few options to try:

1 Headless mode:

It is believed that headless mode is faster. Check this answer Running Selenium with Headless Chrome Webdriver and here Difference of Headless browsers for automation Your code would look like (loading pictures not disabled):

from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

import time
start_time = time.time()
options = webdriver.ChromeOptions()
options.headless = True  # running in headless mode
options.add_argument("--disable-blink-features")
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(chrome_options=options, executable_path='/snap/bin/chromium.chromedriver')

driver.get("https://www.nike.com.br/chuteira-nike-premier-2-sala-unissex-153-169-171-309321")
wait = WebDriverWait(driver, 15)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.cc-allow'))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//label[@for='tamanho__idM40F395']"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button#anchor-acessar-unite-oauth2'))).click()
end_time = time.time()
print("Total execution time: {} seconds".format(end_time - start_time))

Total execution time (3 runs):

12.759594202041626 seconds
13.571481466293335 seconds
17.36514186859131 seconds

2 Disable pictures Reference: Python: Disable images in Selenium Google ChromeDriver

from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

import time
start_time = time.time()
options = webdriver.ChromeOptions()

prefs = {'profile.default_content_setting_values': {'images': 2}}
options.add_experimental_option("prefs", prefs)  # disable loading pictures
options.add_argument("--disable-blink-features")
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(chrome_options=options, executable_path='/snap/bin/chromium.chromedriver')

driver.get("https://www.nike.com.br/chuteira-nike-premier-2-sala-unissex-153-169-171-309321")
wait = WebDriverWait(driver, 15)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.cc-allow'))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//label[@for='tamanho__idM40F395']"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button#anchor-acessar-unite-oauth2'))).click()
end_time = time.time()
print("Total execution time: {} seconds".format(end_time - start_time))

Total execution time: 10.57245922088623 seconds
Total execution time: 13.770843982696533 seconds
Total execution time: 10.908315896987915 seconds

3 Disable CSS Reference How to disable CSS in Python selenium using ChromeOptions - Not sure this option is correct now because it does not improve waiting times and works only in headless mode:

from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

import time
start_time = time.time()
options = webdriver.ChromeOptions()
prefs = {'profile.default_content_setting_values': {'cookies': 2, 'images': 2, 'javascript': 2,
                            'plugins': 2, 'popups': 2, 'geolocation': 2,
                            'notifications': 2, 'auto_select_certificate': 2, 'fullscreen': 2,
                            'mouselock': 2, 'mixed_script': 2, 'media_stream': 2,
                            'media_stream_mic': 2, 'media_stream_camera': 2, 'protocol_handlers': 2,
                            'ppapi_broker': 2, 'automatic_downloads': 2, 'midi_sysex': 2,
                            'push_messaging': 2, 'ssl_cert_decisions': 2, 'metro_switch_to_desktop': 2,
                            'protected_media_identifier': 2, 'app_banner': 2, 'site_engagement': 2,
                            'durable_storage': 2}}
options.add_experimental_option("prefs", prefs)  # disable loading pictures
options.headless = True  # running in headless mode
options.add_argument("--disable-blink-features")
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(chrome_options=options, executable_path='/snap/bin/chromium.chromedriver')

driver.get("https://www.nike.com.br/chuteira-nike-premier-2-sala-unissex-153-169-171-309321")
wait = WebDriverWait(driver, 15)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.cc-allow'))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//label[@for='tamanho__idM40F395']"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button#anchor-acessar-unite-oauth2'))).click()
end_time = time.time()
print("Total execution time: {} seconds".format(end_time - start_time))


Total execution time: 13.649555444717407 seconds
Total execution time: 12.698347806930542 seconds
Total execution time: 13.572114706039429 seconds

Conclusion

From what I see using prefs = {'profile.default_content_setting_values': {'images': 2}} is the optimal solution, at least for your case. Headless mode does not add any improvements in speed.

However, results may be different depending on Selenium code and other factors.

Also, here you can find some good advices on how to make tests faster. List:

Use fast selectors
Use fewer locators
Create atomic tests
Don't test the same functionality twice
Use only explicit waits
Use the chrome driver
Use drivers for headless browsers
Re-use the browser instance
Run scripts in parallel
Use HTTP parse libraries
Pre-populate cookies do not load images in the web page

You are not using implicit waits, using css selectors which are faster.

How to remove CSS loading and Selenium text

1 Answers1