Airflow: how to use xcom_push and xcom_pull in non-PythonOperator

Question

I see a lot of examples on how to use xcom_push and xcom_pull with PythonOperators in Airflow.

I need to do xcom_pull from a non-PythonOperator class and couldn't find how to do it.

Any pointer or example will be appreciated!

score 8 · Accepted Answer · answered Jul 30 '18 at 05:59

8

You can access XCom variables from within templated fields. For example, to read from XCom:

myOperator = MyOperator(
    message="Operation result: {{ task_instance.xcom_pull(task_ids=['task1', 'task2'], key='result_status') }}",
    ...

It is also possible to not specify task to get all XCom pushes within one DagRun with the same key name

myOperator = MyOperator(
    message="Warning status: {{ task_instance.xcom_pull(task_ids=None, key='warning_status') }}",
    ...

would return an array.

answered Jul 30 '18 at 05:59

tobi6

1

This doesn't address pushing from a non-PythonOperator. – dboshardy Jan 09 '19 at 15:49
Correct. In my version of the question, only "I need to do xcom_pull from a non-PythonOperator" was asked. – tobi6 Jan 10 '19 at 08:41

bcb · Answer 2 · 2020-01-06T00:55:49.940

4

Push:

self.xcom_push(context, key, value)

Pull:

self.xcom_pull(context, key=key)

Push:

context["ti"].xcom_push(key, value)

Pull:

context["ti"].xcom_pull(key=key)

edited Jan 06 '20 at 00:55

answered Apr 07 '19 at 23:37

bcb

Would you please elaborate this answer by mentioning, where these lines can be added? – Sandeep Veerlapati Dec 17 '19 at 11:19
1

Ok, added more info. – bcb Dec 18 '19 at 23:43

2 Answers2