selenium firefox Unable to locate window

Question

I am using ssh connection to server and I'm unable to locate window using selenium3.4 and firefox56. couldn't find a solution, noticed it's mostly an IE bug with selenium code: i

mport bs4 as bs
from bs4 import BeautifulSoup
import urllib.request
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
import re
from random import randint
import pandas as pd
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from pyvirtualdisplay import Display


def get_soup(url):
    sauce = urllib.request.urlopen(url)
    return BeautifulSoup(sauce, 'lxml')


def get_driver_soup(url):
    # driver = webdriver.Firefox(executable_path='/usr/bin/geckodriver')
    display = Display(visible=0, size=(800, 600))
    display.start()
    driver = webdriver.Firefox('/var/gecodriver19-64')
    driver.get(url)
    try:
        element = WebDriverWait(driver, 20).until(
            EC.presence_of_element_located((By.CLASS_NAME, "product-image-wrapper"))
        )
    finally:
        soup = BeautifulSoup(driver.page_source, 'lxml')
        time.sleep(randint(30, 70))
        driver.quit()
        return soup

Complete Traceback::

Traceback (most recent call last):
  File "jomashop.py", line 86, in <module>
    soup = get_driver_soup(companies_list[x] + page_suffix)
  File "jomashop.py", line 32, in get_driver_soup
    soup = BeautifulSoup(driver.page_source, 'lxml')
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 587, in page_source
    return self.execute(Command.GET_PAGE_SOURCE)['value']
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 311, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/errorhandler.py", line 237, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchWindowException: Message: Unable to locate window

Normally it's taking to long for the page to load. Or sometimes your ip address can be banned if you make to many calls — johnashu, Dec 10 '17 at 12:22
added traceback. I don't think this is the issue because i've worked with that site before — Amittai Aharoni, Dec 10 '17 at 12:37
Maybe there is extra traffic today.. try a high value wait like 100. To check — johnashu, Dec 10 '17 at 12:39
but are you sure it's not an issue with my code? have you seen the 'nowindowexception' before? — Amittai Aharoni, Dec 10 '17 at 12:41
Yes I have seen the error and it is normally from timing out.. Looking at your code though. the error is in the BeautifulSoup Line.. I have provided a possible answer for you — johnashu, Dec 10 '17 at 12:52

score 0 · Answer 1 · answered Dec 10 '17 at 12:52

0

It is possible that it is timing out when you try to use Selenium Driver to return the source HTML.

Instead of this:

finally:
    soup = BeautifulSoup(driver.page_source, 'lxml')

Try using requests instead:

import requests

finally:
    r = requests.get(url) 
    html_bytes = r.text 
    soup = BeautifulSoup(html_bytes, 'lxml')

This should pull only the html

answered Dec 10 '17 at 12:52

johnashu

2,167
4
19
44

returns :selenium.common.exceptions.WebDriverException: Message: connection refused – Amittai Aharoni Dec 10 '17 at 13:06
That means the server is blocking your IP , at a guess.. The code I posted does not use Selenium so that error is more than likely from the ......WebDriverWait(driver, 20)..... line.. – johnashu Dec 10 '17 at 13:12
have a look here.. maybe this is relevent.. ... https://stackoverflow.com/questions/39547598/selenium-common-exceptions-webdriverexception-message-connection-refused – johnashu Dec 10 '17 at 13:13
WebDriverWait is correct, worked locally. Stopped working on ssh – Amittai Aharoni Dec 10 '17 at 13:20
I think it will be the remote server.. sometimes they can ban you for like 30 mins.. sometimes permanent.. you really need to try the script on a completley different website to be sure its not the server – johnashu Dec 10 '17 at 13:39

score 0 · Answer 2 · answered Dec 10 '17 at 13:15

Check your geckodriver.log file (should be in the same directory as python file)

If it says

Error: GDK_BACKEND does not match available displays

then install pyvirtualdisplay:

 pip install pyvirtualdisplay selenium

You might need xvfb too:

 sudo apt-get install xvfb

Then try adding this code:

from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()

Full example:

from pyvirtualdisplay import Display
from selenium import webdriver

display = Display(visible=0, size=(800, 600))
display.start()

driver = webdriver.Firefox()
driver.get('http://www.python.org')

driver.close()

score 0 · Answer 3 · answered Dec 11 '17 at 12:39

0

The error says it all :

NoSuchWindowException: Message: Unable to locate window

Change the line :

driver = webdriver.Firefox('/var/gecodriver19-64')

To :

driver = webdriver.Firefox(executable_path='/var/gecodriver19-64/geckodriver')

answered Dec 11 '17 at 12:39

undetected Selenium

183,867
41
278
352

selenium firefox Unable to locate window

3 Answers3