Run DAG inside airflow container works fine but gets the AirflowException("Bash command failed") when triggered from Web

Asked Oct 16 '19 at 04:58

Active Oct 16 '19 at 05:24

Viewed 216 times

If I run the DAG inside the docker container with airflow run dag_A task_1 ... command, it works fine. But if I trigger the same DAG via the web-UI, it will fail with the following error: AirflowException: Bash command failed. Sifting through the error logs, revealed this error ImportError: No module named some-lib where some-lib is a local module inside the dag folder. I think the error is related to this question posted in SO - Apache Airflow DAG cannot import local module

I added the __init__.py files in the dag folder and sub-folders but did not seem to help. I also set ENV PYTHONPATH=<path-to-dag>:${PYTHONPATH} in Dockerfile but did not work as well.

edited Oct 16 '19 at 05:24

asked Oct 16 '19 at 04:58

alltej

6,787
10
46
87

Try doing the by manually adding the path to the dag folder in your DAG file using `sys.path.append(os.environ[...])` instead. – absolutelydevastated Oct 16 '19 at 05:51
@absolutelydevastated Can you expand a little bit on this? – alltej Oct 16 '19 at 05:54
Inside your DAG file, do `sys.path.append(path_to_somelib_folder)` then import your local module. – absolutelydevastated Oct 16 '19 at 11:53
Usually the DAGs are in `os.environ['AIRFLOW_HOME']`. – absolutelydevastated Oct 16 '19 at 11:59

Run DAG inside airflow container works fine but gets the AirflowException("Bash command failed") when triggered from Web

0 Answers0