1

Every time I start airflow worker I keep getting this error

[2017-11-07 16:24:12,354: ERROR/MainProcess] consumer: Cannot connect to amqp://myuser:**@127.0.0.1:8793/myvhost: timed out. Trying again in 26.00 seconds...

I have followed the instructions to install CeleryExecutors on Airflow as well as installing RabbitMQ using this documentation.

I have went and configured my airflow.cfg to reflect this by changing the celery_result_backend and broker_url to point to the right address (amqp://myuser:mypassword@localhost:8793/myvhost for example, from the documentation). I had this up and running at some point and when I changed the DAG directory. Changing the DAG directory shouldn't have an effect on it besides changing what's inside the DagBag.

Is there anything else I'm supposed to look at to debug and get the Celery Workers up and running?

Minh
  • 2,180
  • 5
  • 23
  • 50

1 Answers1

0

Following the posts here and here, it was because I was running Celery4.1.0. I downgraded to Celery3.1.7 and it is now working.

Minh
  • 2,180
  • 5
  • 23
  • 50