0

I have an airflow dag that uses the following jinja template: "{{ execution_date.astimezone('Etc/GMT+6').subtract(days=1).strftime('%Y-%m-%dT00:00:00') }}"

This template works in other dags, and it works when the schedule_interval for the dag is set to timedelta(hours=1). However, when we set the schedule interval to 0 8 * * *, it throws the following traceback at runtime:

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/airflow/models/__init__.py", line 1426, in _run_raw_task
    self.render_templates()
  File "/usr/lib/python2.7/site-packages/airflow/models/__init__.py", line 1790, in render_templates
    rendered_content = rt(attr, content, jinja_context)
  File "/usr/lib/python2.7/site-packages/airflow/models/__init__.py", line 2538, in render_template
    return self.render_template_from_field(attr, content, context, jinja_env)
  File "/usr/lib/python2.7/site-packages/airflow/models/__init__.py", line 2520, in render_template_from_field
    for k, v in list(content.items())}
  File "/usr/lib/python2.7/site-packages/airflow/models/__init__.py", line 2520, in <dictcomp>
    for k, v in list(content.items())}
  File "/usr/lib/python2.7/site-packages/airflow/models/__init__.py", line 2538, in render_template
    return self.render_template_from_field(attr, content, context, jinja_env)
  File "/usr/lib/python2.7/site-packages/airflow/models/__init__.py", line 2514, in render_template_from_field
    result = jinja_env.from_string(content).render(**context)
  File "/usr/lib64/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib64/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 1, in top-level template code
TypeError: astimezone() argument 1 must be datetime.tzinfo, not str

It appears the execution date being passed in is a string, not a datetime object; but I am only able to hit this error on this specific dag, and no others. I've tried deleting the dag entirely and recreating it with no luck.

chris.mclennon
  • 966
  • 10
  • 25

1 Answers1

4

Looks like astimezone(..) function is misbehaving, it expects a datetime.tzinfo while you are passing it an str argument ('Etc/GMT+6')

TypeError: astimezone() argument 1 must be datetime.tzinfo, not str

While I couldn't make the exact thing work, I believe following achieves pretty much the same effect as what you are trying

{{ execution_date.in_timezone("US/Eastern") - timedelta(days=1) }}

Recall that

  • execution_date macro is a Pendulum object
  • in_timezone(..) converts it into a datetime.datetime(..)
  • then we just add a datetime.timedelta(days=1) to it
y2k-shubham
  • 10,183
  • 11
  • 55
  • 131
  • Thanks @y2k-shubham, this helped debug the issue experienced. It looks like Airflow inconsistently passes in execution_date as either a Pendulum datetime object or a Python datetime object, depending on whether the schedule_interval is a cron syntax or a timedelta. It appears someone else ran into a similar flavor of this issue: https://issues.apache.org/jira/browse/AIRFLOW-4788 – chris.mclennon Jul 16 '19 at 23:27