3

Many times when I try to open the tree view or task duration page of some DAGs in the UI I get the error: 504 gateway time-out. Sometimes after that I can't even open the page with the list of DAGs. Do you know where this problem could come from?

The CPU and memory of the machine running Airflow seem to be fine and I use RDS for the metadata.

Thanks!

Pierre
  • 938
  • 1
  • 15
  • 37

1 Answers1

3

I've experienced this before as well. I believe it's caused by an HTTP request that takes longer than expected for the webserver's gunicorn worker to fulfill. For example, if you set the DAG tree view to a high setting like 365 DAG runs for a DAG with a lot of tasks, you may be able to reproduce this consistently.

Can you try bumping up the timeout settings on the webserver to see if it makes a difference?

  • First, try increasing web_server_worker_timeout (default = 120 seconds) under the [webserver] group.
  • If that doesn't resolve it, you might also try increasing web_server_master_timeout under the same group.
  • Another technique to try is switching the webserver worker_class (default = sync) to eventlet or gevent.

Reference: https://github.com/apache/incubator-airflow/blob/c27098b8d31fee7177f37108a6c2fb7c7ad37170/airflow/config_templates/default_airflow.cfg#L225-L229

Note that the alternative worker classes require installing Airflow with the async extras like:

pip install apache-airflow[async]

You can find more info about gunicorn worker timeouts in this question: How to resolve the gunicorn critical worker timeout error?.

Taylor D. Edmiston
  • 12,088
  • 6
  • 56
  • 76
  • 1
    Hi @shankshera - I am marking your suggested edit as rejected because [*extras*](https://stackoverflow.com/questions/52474931/what-is-extra-in-pypi-dependency) is the official technical term for an optional sub-dependency in Py (not "subpackage" as it is not actually a sub). As far as adding quotes, you should not need quotes to run this command in a shell — see https://packaging.python.org/tutorials/installing-packages/#installing-setuptools-extras for example. – Taylor D. Edmiston May 19 '19 at 20:20