0

I am trying to add airflow dag dynamically looping through the dictionary keys and assigning keys as dag name.

dags are creating fine but i am getting :"This DAG isn't available in the webserver DagBag object. It shows up in this list because the scheduler marked it as active in the metdata database" and its not clickable.

def create_dag(dag_id):
    args = build_default_args(config_file)
    dag = DAG(dag_id,schedule_interval='30 11 * * *', default_args=args)
    with dag:
        init_task = BashOperator(
                    task_id='test_init_task',
                    bash_command='echo "task"',
                    dag=dag
                )
        init_task
    return dag

def get_data(**kwargs):
    my_list=[]
    file = open("/home/airflow/gcs/data/test.json")
    data=json.load(file)
    return data

data1 = data()

for dict in data1:
    for pair in dict.items():
        key , value = pair
        print "key",ls_table ,"value",metrics
        dag_id = '{}'.format(key)
        default_args = {'owner': 'airflow',
                    'start_date': datetime(2019, 6, 18)
                    }
        schedule = '@daily'
        globals()[dag_id] = create_dag(dag_id)
user8617180
  • 267
  • 6
  • 20
  • Is it absolutely necessary to create DAGs? Can the same output be achieved by creating tasks inside a DAG? – nightgaunt Jun 20 '19 at 06:52
  • check out this post: https://stackoverflow.com/questions/41560614/airflow-this-dag-isnt-available-in-the-webserver-dagbag-object – David Schuler Jun 20 '19 at 16:58

0 Answers0