13

Airflow example dags remain in the UI even after I have turned off load_examples = False in config file.

enter image description here

The system informs the dags are not present in the dag folder but they remain in UI because the scheduler has marked it as active in the metadata database.

I know one way to remove them from there would be to directly delete these rows in the database but off course this is not ideal.How should I proceed to remove these dags from UI?

fernandosjp
  • 2,658
  • 1
  • 25
  • 29

4 Answers4

10

There is currently no way of stopping a deleted DAG from being displayed on the UI except manually deleting the corresponding rows in the DB. The only other way is to restart the server after an initdb.

Him
  • 1,609
  • 12
  • 20
5

Airflow 1.10+:

  • Edit airflow.cfg and set load_examples = False
  • For each example dag run the command airflow delete_dag example_dag_to_delete

This avoids resetting the entire airflow db.

(Since Airflow 1.10 there is the command to delete dag from database, see this answer )

Ena
  • 3,481
  • 36
  • 34
2

Assuming you have installed airflow through Anaconda. Else look for airflow in your python site-packages folder and follow the below.

After you follow the instructions https://stackoverflow.com/a/43414326/1823570

  • Go to $AIRFLOW_HOME/lib/python2.7/site-packages/airflow directory
  • Remove the directory named example_dags or just rename it to revert back
  • Restart your webserver

cat $AIRFLOW_HOME/airflow-webserver.pid | xargs kill -9

airflow webserver -p [port-number]

Community
  • 1
  • 1
ghosts
  • 177
  • 2
  • 15
  • Won't `airflow initdb` restart the whole database? This airflow instance is already in production and I already have history data from the Dags which are running. I was looking for a solution that would fix the issue but trying to avoid restarting from scratch. – fernandosjp Jun 28 '17 at 21:33
  • I think you could avoid that step. Haven't tried though. – ghosts Jun 29 '17 at 00:59
  • Yeah, me too. Followed similar approach like yours. – ghosts Sep 06 '18 at 23:15
  • Hey, `airflow resetdb` cleaned up all my jobs but, I am seeing error in the UI after running resetdb. I am not able to access anything in the UI. Please check: https://stackoverflow.com/questions/52675779/which-version-of-mysql-is-compatible-with-airflow-version-1-10 – hadooper Oct 06 '18 at 05:52
0

Definitely airflow resetdb works here.

What I do is to create multiple shell scripts for various purposes like start webserver, start scheduler, refresh dag, etc. I only need to run the script to do what I want. Here is the list:

(venv) (base) [pchoix@hadoop02 airflow]$ cat refresh_airflow_dags.sh
#!/bin/bash
cd ~
source venv/bin/activate
airflow resetdb
(venv) (base) [pchoix@hadoop02 airflow]$ cat start_airflow_scheduler.sh
#!/bin/bash
cd /home/pchoix
source venv/bin/activate
cd airflow
nohup airflow scheduler >> "logs/schd/$(date +'%Y%m%d%I%M%p').log" &
(venv) (base) [pchoix@hadoop02 airflow]$ cat start_airflow_webserver.sh
#!/bin/bash
cd /home/pchoix
source venv/bin/activate
cd airflow
nohup airflow webserver >> "logs/web/$(date +'%Y%m%d%I%M%p').log" &
(venv) (base) [pchoix@hadoop02 airflow]$ cat start_airflow.sh
#!/bin/bash
cd /home/pchoix
source venv/bin/activate
cd airflow
nohup airflow webserver >> "logs/web/$(date +'%Y%m%d%I%M%p').log" &
nohup airflow scheduler >> "logs/schd/$(date +'%Y%m%d%I%M%p').log" &

Don't forget to chmod +x to those scripts

I hope you find this helps.

mdivk
  • 3,545
  • 8
  • 53
  • 91