1

In the Airflow web interface I can view a list of DAGs. If I click on one of those DAGs I can see different views of the DAG or the code for the DAG, and in the details tab I can see the name of the DAGs file, but not the path.

I've also tried airflow list-dags but that doesn't provide the information I'm looking for either.

There is a DAG I can see in the web interface but I can't find where it is in the filesystem.

Is there a way to find the location of a DAG from the Airflow web interface or the Airflow CLI?

Gregory Arenius
  • 2,904
  • 5
  • 26
  • 47

5 Answers5

6

On the main DAGs list in the ui, there is a button to the left of the DAG pause toggle. It resolves to:

https://<airflow.blah.com>/admin/dagmodel/edit/?id=<dag_id>

The Fileloc will show you where airflow thinks your DAG file is. However...

  • it can be wrong if you import other local files into you DAG file the
  • the file may not be there anymore, this info is stored in the db
marengaz
  • 1,639
  • 18
  • 28
1

via CLI: airflow config get-value core DAGS_FOLDER

Ben
  • 1,133
  • 1
  • 15
  • 30
  • While this answer may be useful, you can improve it by saying why it works, how it works, when it should be used, and what its limitations are. Please [edit] your answer to include explanation and link to relevant documentation. – Dorian Turba Feb 09 '23 at 11:50
0

i don't think so, but you should just search your dags_folder from your airflow.cfg to find where dags are created.

additionally, the logs for airflow-webserver tell you exactly which files/dags are being imported, assuming you have some logging on.

acushner
  • 9,595
  • 1
  • 34
  • 34
  • There are DAGs that aren't in the dags_folder. – Gregory Arenius Jul 31 '17 at 20:10
  • even subdirectories of it? regardless, the webserver logs have every dag that's being imported. additionally, you can have dags that were once run but no longer found but they'll look different in the web UI. – acushner Jul 31 '17 at 20:12
0

Especially if you do development and change your DAGs / delete the DAG files, the entries will often persist in the backend DB and be visible in the UI. Most of the time, there isn't actually any DAG file anymore.

I'd recommend cleaning the DAGs from the database, using something like:

import sys
from airflow.hooks.postgres_hook import PostgresHook

dag_input = sys.argv[1]
hook=PostgresHook( postgres_conn_id= "airflow_db")

for t in ["xcom", "task_instance", "sla_miss", "log", "job", "dag_run", "dag" ]:
  sql="delete from {} where dag_id='{}'".format(t, dag_input)
  hook.run(sql, True)

(Based on an answer from Airflow: how to delete a DAG?)

Ladislav Indra
  • 819
  • 6
  • 10
0

Airflow by default provides some sample DAGs. The DAGs are kept under the example_dags folder.

Path of the DAGs: lib/python2.7/site-packages/airflow/example_dags

Neil
  • 821
  • 9
  • 17