11

I am passing arguments to a Taskflow based Dag using the REST API. Looking at similar issues raised on this forum, it seems below are the usual ways of accessing the passed arguments.

#From inside a template field or file:
{{ dag_run.conf['key'] }}
#Or when context is available, e.g. within a python callable of the PythonOperator:
context['dag_run'].conf['key']

I attempted to get the context dictionary

@dag(default_args=default_args, schedule_interval=None, start_date=days_ago(2), params=None)
def classic_xgb(**context):
    """
    ### TaskFlow API Tutorial Documentation
    [here](https://airflow.apache.org/docs/stable/tutorial_taskflow_api.html)
    """
    @task()
    def extract():
        print("context is ", context)

The output is <airflow.models.dagparam.DagParam object at 0x7f735c634510> Now how do I get to the conf dictionary that was passed as the input arguement to the Dag? I need to use the arguements in my python code and so templating option does not seem viable for me.

Any help would be most appreciated.

Thanks

Best Regards,

Adeel

Adeel Hashmi
  • 767
  • 1
  • 8
  • 20

1 Answers1

22

There is a new function get_current_context() to fetch the context in Airflow 2.0. Once you have the context dict, the 'params' key contains the arguments sent to the Dag via REST API. The following code solved the issue.

from airflow.operators.python import task, get_current_context

default_args = {
    'owner': 'airflow',
}
@dag(default_args=default_args, schedule_interval=None, start_date=days_ago(2))
def classic_xgb(**kwargs):

    """
    @task()
    def extract():
        context = get_current_context()
SergiyKolesnikov
  • 7,369
  • 2
  • 26
  • 47
Adeel Hashmi
  • 767
  • 1
  • 8
  • 20
  • 8
    Thank you, this really needs to be better documented. – dead10ck Oct 07 '21 at 18:03
  • Say I want to access not the argument sent to the DAG via REST API, but instead a template Airflow variable such as `logical_date`, or `ds`. How would I do so? – Diana Vazquez Romo Aug 12 '22 at 19:16
  • This is documented a bit better now: https://airflow.apache.org/docs/apache-airflow/stable/tutorial_taskflow_api.html#accessing-context-variables-in-decorated-tasks – ZaxR Sep 10 '22 at 20:44