"Failed to read marionette port" in Selenium + firefox + geckodriver + python from php

Question

I´m running a python script from php where I do some scrapping. The script runs ok when executed on Ubuntu terminal, but gives me these lines on Apache log:

df=BUscrap(sys.argv[1])
  File "/var/www/scrapbot.chambeala.com/./realScrapper.py", line 261, in BUscrap
    driver=webdriver.Firefox(service=service, options=options)
  File "/var/www/myenv/lib/python3.10/site-packages/selenium/webdriver/firefox/webdriver.py", line 68, in __init__
    super().__init__(command_executor=executor, options=self.options)
  File "/var/www/myenv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 206, in __init__
    self.start_session(capabilities)
  File "/var/www/myenv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 291, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
  File "/var/www/myenv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 346, in execute
    self.error_handler.check_response(response)
  File "/var/www/myenv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Failed to read marionette port

This is the part in Python where I set and start Geckodriver:

def BUscrap(limit=""):
    limit=int(limit)
    print("Capturando nuevos trabajos...")
    options=webdriver.FirefoxOptions()
    service = Service(executable_path='/usr/local/bin/geckodriver')
    options.add_argument('--headless')
    options.add_argument('--no-sandbox')
    options.add_argument('--disable-dev-shm-usage')
    options.add_argument(f'--proxy-server={proxy_server_url}')
    options.add_argument('user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.50 Safari/537.36')
    driver=webdriver.Firefox(service=service, options=options)
    driver.implicitly_wait(10)

And this is the call from PHP:

<?php
$cmd = escapeshellcmd("./realScrapper.py " .$_POST["ligas"]);
if(isset($_POST['ejecutar'])) {
    echo '<div class="output">';
    while (@ ob_end_flush());
    $proc = popen($cmd, 'r');
    echo '<pre>';
    while (!feof($proc))
    {
        echo fread($proc, 4096);
        @ flush();
    }
    echo '</pre>';
    $_POST = array();
    echo '</div>';
}
?>

Firefox and Geckodriver have execution permissions, and PHP code works fine with other scripts. Firefox wasn't installed through snap, but from a debian package.

Python version is 3.10.6, Selenium version is 4.10.0, Firefox version is 107.0.1, Geckodriver version is 0.33.0

Let me know if more information is needed.

I was trying to execute a Python scrapping script from PHP, but the web page got idle and then stopped.

score 0 · Answer 1 · answered Jul 31 '23 at 23:08

The following arguments are ChromeDriver specific arguments and aren't applicable to GeckoDriver so you may remove them:

options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument(f'--proxy-server={proxy_server_url}')

Additionally replace:

options.add_argument('--headless')

with:

options.add_argument('--headless=new')

as DeprecationWarning: headless property is deprecated, instead use add_argument('--headless') or add_argument('--headless=new') on Selenium 4.8.0 Python and execute your test.

Thank you, I removed those lines and the scrapper still works on terminal. --headless=new got an unexpected exit exception though. Unfortunately, the code still doesn't work from php. — Arturo Domnico Navarro Riveros, Aug 01 '23 at 04:11

"Failed to read marionette port" in Selenium + firefox + geckodriver + python from php

1 Answers1