1

I'm very new to airflow.

I schedule a dag file to run every 10 minutes. My default args are below.

args = {
    'owner': 'crv',
    'start_date': datetime(2021, 1, 1),
}

dag = DAG(
    dag_id='score',
    default_args=args,
    schedule_interval='*/10 * * * *',
    tags=['example']
)

I ran airflow scheduler on background using following command.

airflow scheduler &

After I execute airflow scheduler & command dag file executed properly and ran the process according to the schedule. So, I exit from the terminal (I was running dag file on a server). After a while I log in to the server and found that dag file isn't running according to the schedule and it stopped running the process after I exit from the terminal. Then I kill currently running scheduler and restart it. After that dag file worked properly. But when I exit from the terminal it stop running the process according to the schedule again.

I checked all the points in this answer https://stackoverflow.com/a/49047832/7909792

My Task Instance Details are below

enter image description here

Can anyone help with my issue. Thank you.

CRV
  • 69
  • 8

1 Answers1

1

You need to run the scheduler in daemon mode. Use airflow scheduler -D.

There are more optional arguments you can read about it in the documentation

Elad Kalif
  • 14,110
  • 2
  • 17
  • 49