Clicking buttons and filling forms with Selenium and PhantomJS

Question

I have a simple task that I want to automate. I want to open a URL, click a button which takes me to the next page, fills in a search term, clicks the "search" button and prints out the url and source code of the results page. I have written the following.

from selenium import webdriver
import time

driver = webdriver.PhantomJS()
driver.set_window_size(1120, 550)

#open URL
driver.get("https://www.searchiqs.com/nybro/")
time.sleep(5)

#click Log In as Guest button
driver.find_element_by_id('btnGuestLogin').click()
time.sleep(5)

#insert search term into Party 1 form field and then search
driver.find_element_by_id('ContentPlaceholder1_txtName').send_keys("Moses")
driver.find_element_by_id('ContentPlaceholder1_cmdSearch').click()
time.sleep(10)

#print and source code
print driver.current_url
print driver.page_source
driver.quit()

I am not sure where I am going wrong but I have followed a number of tutorials on how to click buttons and fill forms. I get this error instead.

Traceback (most recent call last):                                                                                                                                              
  File "phant.py", line 12, in <module>                                                                                                                                         
    driver.find_element_by_id('btnGuestLogin').click()                                                                                                                          
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 269, in find_element_by_id                                                         
    return self.find_element(by=By.ID, value=id_)                                                                                                                               
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 752, in find_element                                                               
    'value': value})['value']                                                                                                                                                   
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute                                                                    
    self.error_handler.check_response(response)                                                                                                                                 
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response                                                          
    raise exception_class(message, screen, stacktrace)                                                                                                                          
selenium.common.exceptions.NoSuchElementException: Message: {"errorMessage":"Unable to find element with id 'btnGuestLogin'","request":{"headers":{"Accept":"application/json","
Accept-Encoding":"identity","Connection":"close","Content-Length":"94","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:35670","User-Agent":"Python-urllib/2.7"
},"httpVersion":"1.1","method":"POST","post":"{\"using\": \"id\", \"sessionId\": \"d38e5fa0-5349-11e6-b0c2-758ad3d2c65e\", \"value\": \"btnGuestLogin\"}","url":"/element","urlP
arsed":{"anchor":"","query":"","file":"element","directory":"/","path":"/element","relative":"/element","port":"","host":"","password":"","user":"","userInfo":"","authority":""
,"protocol":"","source":"/element","queryKey":{},"chunks":["element"]},"urlOriginal":"/session/d38e5fa0-5349-11e6-b0c2-758ad3d2c65e/element"}}                                  
Screenshot: available via screen

The error seems to suggest that the element with that id does not exist yet it does.

--- EDIT: Changed code to use WebDriverWait ---

I have changed some things around to implement WebDriverWait

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

driver = webdriver.PhantomJS()
driver.set_window_size(1120, 550)

#open URL
driver.get("https://www.searchiqs.com/nybro/")

#click Log In as Guest button
element = WebDriverWait(driver, 10).until(
            EC.presence_of_element_located((By.ID, "btnGuestLogin"))
        )
element.click()

#wait for new page to load, fill in form and hit search
element2 = WebDriverWait(driver, 10).until(
            EC.presence_of_element_located((By.ID, "ContentPlaceholder1_cmdSearch"))
        )
#insert search term into Party 1 form field and then search
driver.find_element_by_id('ContentPlaceholder1_txtName').send_keys("Moses")
element2.click()
driver.implicitly_wait(10)

#print and source code
print driver.current_url
print driver.page_source
driver.quit()

It still raises this error

Traceback (most recent call last):                                                                                                                                              
  File "phant.py", line 14, in <module>                                                                                                                                         
    EC.presence_of_element_located((By.ID, "btnGuestLogin"))                                                                                                                    
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/support/wait.py", line 80, in until                                                                           
    raise TimeoutException(message, screen, stacktrace)                                                                                                                         
selenium.common.exceptions.TimeoutException: Message:                                                                                                                           
Screenshot: available via screen

See if there is any frame around...or need to implement `WebDriverWait` to wait until element visible and clickable instead of `time.sleep` ...:) — Saurabh Gaur, Jul 26 '16 at 16:18
@SaurabhGaur, I have made the edit, you can see them at the bottom of the post with the new alternative script. I still get the error — Tendekai Muchenje, Jul 26 '16 at 16:39
And what about frame??... make sure `btnGuestLogin` is not inside any frame or iframe — Saurabh Gaur, Jul 26 '16 at 16:44
@SaurabhGaur , it's not in a frame. I inspected the page source, there are no frames. — Tendekai Muchenje, Jul 26 '16 at 17:02

score 2 · Accepted Answer · edited May 23 '17 at 12:08

2

The WebDriverWait approach actually works for me as is:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


driver = webdriver.PhantomJS()
driver.set_window_size(1120, 550)

driver.get("https://www.searchiqs.com/nybro/")

element = WebDriverWait(driver, 10).until(
            EC.presence_of_element_located((By.ID, "btnGuestLogin"))
        )
element.click()

No errors. PhantomJS version 2.1.1, Selenium 2.53.6, Python 2.7.

The issue might be related to SSL and PhantomJS, either work through http:

driver.get("http://www.searchiqs.com/nybro/")

Or, try ignoring SSL errors:

driver = webdriver.PhantomJS(service_args=['--ignore-ssl-errors=true', '--ssl-protocol=any'])

edited May 23 '17 at 12:08

Community

1
1

answered Jul 26 '16 at 17:03

alecxe

462,703
120
1,088
1,195

@TendekaiMuchenje okay, print the page source, what do you get? `print(driver.page_source)`. – alecxe Jul 26 '16 at 17:49
if i put it right after doing `driver.get()`, it prints out `` , anywhere after that results in the error – Tendekai Muchenje Jul 26 '16 at 18:19
@TendekaiMuchenje okay, see you have the empty page source. Try adding the `'--ssl-protocol=any'` argument as well, updated the answer. – alecxe Jul 26 '16 at 18:22
I still get the same error. I am not sure why. I switched in "www.google.com" as my site, and it still gave me the empty page source as well. – Tendekai Muchenje Jul 27 '16 at 00:30
@TendekaiMuchenje have you tried to reinstall/install the latest phantomjs? Do you use any proxies in regular browsers? – alecxe Jul 27 '16 at 01:24
I am now spinning a new Ubuntu machine on Azure and reinstalling everything from scratch. Hopefully that should work. I'll install your configuration so that we are on the same page. And to answer your other question, no I don't use proxies. – Tendekai Muchenje Jul 27 '16 at 01:32
1

IT WORKED! Thanks for the help! – Tendekai Muchenje Jul 27 '16 at 06:30

Clicking buttons and filling forms with Selenium and PhantomJS

1 Answers1