4

So I have written this python code which is supposed to update my advertisment on a german roomshare website automatically:

from time import sleep
from selenium import webdriver
import sched, time
from selenium.webdriver import FirefoxOptions
import datetime


s = sched.scheduler(time.time, time.sleep)


def click_button_by_id(button_s, browser):
    button = browser.find_element_by_id(button_s)
    button.click()



def refresh_ads(sc):

    opts = webdriver.ChromeOptions()
    opts.add_argument("--headless")
    opts.add_argument("--no-sandbox")
    opts.add_argument("--disable-dev-shm-usage")
    browser = webdriver.Chrome(options = opts)
    browser.get('https://www.wg-gesucht.de')
    try:
        click_button_by_id("cmpbntyestxt", browser)
    except:
        pass

    browser.implicitly_wait(5)
    login_link = browser.find_element_by_xpath('//*[@id="headbar_wrapper"]/div[2]/a[2]')
    login_link.click()

    sleep(2)

    username_input = browser.find_element_by_id("login_email_username")
    password_input = browser.find_element_by_id("login_password")

    username_input.send_keys("MY_USERNAME")
    password_input.send_keys("MY_PASSWORD")

    login_button = browser.find_element_by_id("login_submit")
    login_button.click()

    sleep(2)

    browser.get('https://www.wg-gesucht.de/angebot-bearbeiten.html?action=update_offer&offer_id=xxxxxxxx')
    click_button_by_id("set_today_date",browser)
    click_button_by_id("update_offer",browser)

    sleep(2)

    browser.get('https://www.wg-gesucht.de/angebot-bearbeiten.html?action=update_offer&offer_id=xxxxxxxx')
    click_button_by_id("set_today_date",browser)
    click_button_by_id("update_offer",browser)

    print("Refreshed adds at:",datetime.datetime.now().strftime("%c"))

    sleep(2)

    browser.close()

    s.enter(1800, 1, refresh_ads, (sc,))

s.enter(1, 1, refresh_ads, (s,))
s.run()

It works completely fine on my homemachine using this:

Ubuntu 20.04
Python 3.8.5
Chromium 88.0.4324.96
Chromedriver 1:85.0.4183.83-0ubuntu0.20.04.2

While crashing on my Server using this:

Ubuntu 18.04
Python 3.8.7 (was 3.6.9 before. I updated)
Chromium 87.0.4280.66
Chromedriver 87.0.4280.66-0ubuntu0.18.04.1

and this error message:

  File "wg_gesucht_bot.py", line 66, in <module>
    s.run()
  File "/usr/lib/python3.6/sched.py", line 154, in run
    action(*argument, **kwargs)
  File "wg_gesucht_bot.py", line 23, in refresh_ads
    browser = webdriver.Chrome(options = opts)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
    desired_capabilities=desired_capabilities)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created
from tab crashed
  (Session info: headless chrome=87.0.4280.66)

I have already tried using Firefox, it was actually what I used in the first place, and is also the reason why I still import FirefoxOptions. Because otherwise it won't work properly (it starts but doesnt get far on the webpage itself) but that is a problem I can solve afterwards.

There was this: https://stackoverflow.com/a/40379664/10811865 but updating to a 5 year old version didn't make a whole lot of sense to me

And this: https://stackoverflow.com/a/59191325/10811865 Which I also tried.

Any help is appreciated

SirHawrk
  • 610
  • 1
  • 4
  • 17
  • Have you tried [phantomjs](https://phantomjs.org/) browser ? – Jawad Jan 27 '21 at 00:37
  • 2
    @Jawad I did not try that no. But I found a different solution, which I posted as an answer. Out of curiosity; Why would I use a deprecated browser? There surely must be JS browsers that are still updated – SirHawrk Jan 27 '21 at 08:41

1 Answers1

4

So I found a solution for the problem:

First of all I had to update python to 3.8.5 which I did using this :

https://linuxize.com/post/how-to-install-python-3-8-on-ubuntu-18-04/

Afterwards I would encounter a no module named selenium found problem so I followed this:

https://shashanksrivastava.medium.com/how-to-fix-no-module-named-selenium-error-in-python-3-da3fd7b61485 to install in manually

Then I still encountered an error described here:

WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser

Those fixes I had already implemented so they weren't a whole lot of help but killing all chromium and chromedriver processes on my machine fixed the problem for me.

And long last I had to add this line of code:

opts.add_argument('window-size=1920x1080');

so that my code would run properly even in headless mode.

SirHawrk
  • 610
  • 1
  • 4
  • 17