Selenium python random Jenkins pipeline fails

Question

I have a problem I cannot figure out how to solve as it occurs randomly.

I have a webpage on which I want to check always if everything is working, such as buttons, videos etc.

So I start using selenium and everything works just fine when I run it locally.

S I decided to automate those test by creating a Jenkins pipeline, and here is the weird thing, out of 16 pipelines, I have always random 2 pipeline that fails and times out always at the same point.

Let me give a bit more details:

one step that all the pipelines have in common is this:

Play video
(wait for the video to finish)
Press Next button

This bit of code is as follow:

# Play Button
time.sleep(5)
wait.until(EC.presence_of_element_located((By.XPATH, "//div[contains(text(),'Play')]"))).click()
time.sleep(120)

# After Video Ends, click Next button
wait.until(EC.presence_of_element_located((By.XPATH, "//button[contains(text(),'Next')]"))).click(

I am not an expert with Selenium, so this is the solution that I could think of. The video plays just fine, always..but randomly it fails because it cannot find the Next button.

On step to debug this, is to take a screenshot and see what is happening. In the screenshot I could see the button, which mean the page was fully loaded. So I decided to increase the web driver wait time to 150sec and yet the same error occurs randomly.

So I was wondering what else I can do to prevent those False Positive from happening?

I hope I made my point clear enough and please if I missed something or need more infos, just ask me.

UPDATE: Thank you Tony for your help. Just to make sure I understood the logic. This is the updated code.

driver.set_window_size(1920,994)
short_timeout = 100
long_timeout = 150

play = None

try:
    next = WebDriverWait(driver, short_timeout).until(EC.element_to_be_clickable((By.XPATH, "//div[contains(text(),'Play')]")))
except TimeoutException:
    try:
        driver.get(driver.current_url)
        next = WebDriverWait(driver, long_timeout).until(EC.element_to_be_clickable((By.XPATH, "//div[contains(text(),'Play')]")))
    except TimeoutException:
        pass

if next is not None:
    driver.execute_script("arguments[0].scrollIntoView();", play)
    driver.execute_script("arguments[0].click();", play) 

# After Video Ends, click Next button

next = None

try:
    next = WebDriverWait(driver, short_timeout).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(),'Next')]")))
except TimeoutException:
    try:
        driver.get(driver.current_url) # reload the current page
        next = WebDriverWait(driver, long_timeout).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(),'Next')]")))
    except TimeoutException:
        pass

if next is not None:
    driver.execute_script("arguments[0].scrollIntoView();", next)
    driver.execute_script("arguments[0].click();", next)

the Try and except must be applied to every action I want to perform on that page, and if it timeout, it reload the page and try again.

There is only one thing that I am not sure I understood. The if statement for the scroll and click. I have to declare it for each try?

score 0 · Answer 1 · answered Sep 29 '21 at 11:20

0

It's difficult to say without having a minimal example to test with.

You shouldn't need your time.sleep() calls and instead use the ExpectedConditions fully. You're already using presence_of_element_located but as the docs state, this check doesn't ensure that the element is visible; and thus clickable.

An expectation for checking that an element is present on the DOM of a page. This does not necessarily mean that the element is visible.

Try using element_to_be_clickable which should check visibility and clickability:

An Expectation for checking an element is visible and enabled such that you can click it.

Also, be careful reusing your WebDriverWait object as they chain.

answered Sep 29 '21 at 11:20

Lucan

2,907
2
16
30

I am so sorry and I understand when you say that is hard to reproduce without any test. Sorry again for the poor request of help. I changed the element to be clickable. Forgive me for asking this. So based on what you said. The wait is stackable, which mean it will keep adding 100secs for each `wait` that I declare right? So if I keep `wait = WebDriverWait(driver,150)` and in my code I change the `wait.until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(),'Next')]"))).click()` to `wait.implicitly_wait(0)` it will use just the 150 of the driver, right? – Nayden Van Sep 29 '21 at 11:39
Yes, I believe so – Lucan Sep 29 '21 at 13:49

Tony · Answer 2 · 2021-10-04T21:38:50.193

0

I would suggest the following:

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By

next = None
try:
    next = WebDriverWait(driver, short_timeout).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(),'Next')]")))                            
except TimeoutException:
    try:
        driver.get(driver.current_url) # reload the current page
        next = WebDriverWait(driver, long_timeout).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(),'Next')]")))
    except TimeoutException:
        pass

if next is not None:
    driver.execute_script("arguments[0].scrollIntoView();", next)
    driver.execute_script("arguments[0].click();", next)

Other option is to use next.click() as you did but I learned that this sometimes doesn't work while executing a click via JavaScript always works.

edited Oct 04 '21 at 21:38

answered Sep 29 '21 at 13:54

Tony

7,767
2
22
51

Thank you so much mate, I did update my post with your code and asked for a question there as I am not sure I understood that bit – Nayden Van Sep 29 '21 at 14:13
See updated answer. You could try the `next.click()` if it works as it is much shorter. Also I am not sure if you need to scroll down, this is just to be sure your element is visible and clickable. The code was for the `next` button as you said it gives the problems so you don't need to apply it to every element click. It's up to you. – Tony Sep 29 '21 at 14:23
Hey buddy, I tried your solution and configured the same thing for the Play button. When selenium reach that point, it just doesn't execute the `click`. Am I missing something? the if statement should go be before the try? Because the try works, at the timeout it refreshes the page but yet does not click – Nayden Van Sep 30 '21 at 09:59
Try to create a sample code locally on your computer outside the pipeline ands debug and see if the `next` is not None, otherwise it will get to the exception to `pass`so nothing will happen. If next is not found then what is the reason for that? It is difficult to say without the full code and without knowing the structure of the page – Tony Oct 04 '21 at 21:44

Selenium python random Jenkins pipeline fails

2 Answers2