set_nonblocking() on a file object with no setblocking() method (Windows pipes don't support non-blocking I/O) with ChromeDriver Selenium CeleryPython

Question

I initialized a webdriver object with celery but reported this error on windows.

set_nonblocking() on a file object with no setblocking() method (Windows pipes don't support non-blocking I/O)

Code trial:

celery_app = Celery()
celery_app.config_from_object('config.celeryconfig')
from celery import Task
from selenium import webdriver
eventlet.monkey_patch(os=False)



class GetDriver(Task):
    test = 'test'
    chromedriver_path = "chromedriver.exe"

    driver = webdriver.Chrome(executable_path=chromedriver_path)

@celery_app.task(base=GetDriver, bind=True)
def demo(self, params):
    print(params)
    print(self.test)

Can cellery initialize a webdriver object at initialization?

score 0 · Answer 1 · edited Nov 26 '19 at 11:56

celery_app = Celery()
celery_app.config_from_object('config.celeryconfig')
from celery import Task
from selenium import webdriver
eventlet.monkey_patch(os=False)



class GetDriver(Task):
    test = 'test'
    chromedriver_path = "chromedriver.exe"

    driver = webdriver.Chrome(executable_path=chromedriver_path)

@celery_app.task(base=GetDriver, bind=True)
def demo(self, params):
    print(params)
    print(self.test)

score 0 · Answer 2 · answered Nov 26 '19 at 12:26

This error message...

set_nonblocking() on a file object with no setblocking() method (Windows pipes don't support non-blocking I/O)

...is observed when the python-2.7 interpreter is unable to locate the ChromeDriver binary while trying to initiate/spawn a new Browsing Context i.e. Chrome Browser session.

In your code block you have used:

chromedriver_path = "chromedriver.exe"
driver = webdriver.Chrome(executable_path=chromedriver_path)

Your program is unable to locate the chromedriver.exe. Hence the error.

Solution

You need to mention the absolute path of the ChromeDriver as follows:

chromedriver_path = r'C:\path\to\chromedriver.exe'
driver = webdriver.Chrome(executable_path=chromedriver_path)

References

You can find a couple of relevant discussions in:

Outro

Non-blocking read on a subprocess.PIPE in python

score -1 · Answer 3 · answered Nov 27 '19 at 03:14

Did you add the parameter "-P eventlet" when starting the "celery" service. I encountered the same problem, and then I removed the parameter and the problem was solved.

celery -A proj worker -l info  -P eventlet   （old）
celery -A proj worker -l info

set_nonblocking() on a file object with no setblocking() method (Windows pipes don't support non-blocking I/O) with ChromeDriver Selenium CeleryPython

3 Answers3

Solution

References

Outro