0

*While upgrading Airflow version 2.0.0 to 2.2.5 got an error in initdb. the log is attached below. *

  • category=DeprecationWarning,
  • /usr/local/lib/python3.6/site-packages/airflow/configuration.py:361: DeprecationWarning: The logging_level option in [core] has been moved to the logging_level option in [logging] - the old setting has been used, but please update your config.
  • option = self._get_option_from_config_file(deprecated_key, deprecated_section, key, kwargs, section)
  • /usr/local/lib/python3.6/site-packages/airflow/configuration.py:361 DeprecationWarning: The hide_sensitive_variable_fields option in [admin] has been moved to the hide_sensitive_var_conn_fields option in [core] - the old setting has been used, but please update your config.
  • /usr/local/lib/python3.6/site-packages/airflow/configuration.py:361 DeprecationWarning: The base_log_folder option in [core] has been moved to the base_log_folder option in [logging] - the old setting has been used, but please update your config.
  • /usr/local/lib/python3.6/site-packages/airflow/configuration.py:361 DeprecationWarning: The default_queue option in [celery] has been moved to the default_queue option in [operators] - the old setting has been used, but please update your config.
  • /usr/local/lib/python3.6/site-packages/airflow/configuration.py:361 DeprecationWarning: The statsd_on option in [scheduler] has been moved to the statsd_on option in [metrics] - the old setting has been used, but please update your config.
  • /usr/local/lib/python3.6/site-packages/airflow/configuration.py:361 DeprecationWarning: The default_queue option in [celery] has been moved to the default_queue option in [operators] - the old setting has been used, but please update your config.
  • DB: postgresql+psycopg2://airflow-dev-ipc:***@7a2dcbf5-4b02-4462-9ab2-f52f835fd961.c7e06sed0lktba7pbqj0.databases.appdomain.cloud:31604/airflow_dev_ipc
  • [2022-12-15 09:36:38,662] {db.py:919} INFO - Creating tables
  • INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
  • INFO [alembic.runtime.migration] Will assume transactional DDL.
  • WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.amazon.aws.hooks.s3.S3Hook' from 'apache-airflow-providers-amazon' package: deprecated() got an unexpected keyword argument 'name'
  • WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook' from 'apache-airflow-providers-amazon' package: deprecated() got an unexpected keyword argument 'name'
  • WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.amazon.aws.hooks.emr.EmrHook' from 'apache-airflow-providers-amazon' package: deprecated() got an unexpected keyword argument 'name'
  • WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.amazon.aws.hooks.s3.S3Hook' from 'apache-airflow-providers-amazon' package: deprecated() got an unexpected keyword argument 'name'
  • WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook' from 'apache-airflow-providers-amazon' package: deprecated() got an unexpected keyword argument 'name'
  • WARNI [airflow.providers_manager] Exception when importing 'airflow.providers.amazon.aws.hooks.emr.EmrHook' from 'apache-airflow-providers-amazon' package: deprecated() got an unexpected keyword argument 'name'
  • INFO [alembic.runtime.migration] Running upgrade c8ffec048a3b -> a56c9515abdc, Remove dag_stat table
  • Traceback (most recent call last):
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
  • cursor, statement, parameters, context
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
  • cursor.execute(statement, parameters)
  • psycopg2.errors.UndefinedTable: table "dag_stats" does not exist
  • The above exception was the direct cause of the following exception:
  • Traceback (most recent call last):
  • File "/usr/local/bin/airflow", line 8, in
  • sys.exit(main())
  • File "/usr/local/lib/python3.6/site-packages/airflow/main.py", line 48, in main
  • args.func(args)
  • File "/usr/local/lib/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, in command
  • return func(*args, **kwargs)
  • File "/usr/local/lib/python3.6/site-packages/airflow/cli/commands/db_command.py", line 31, in initdb
  • db.initdb()
  • File "/usr/local/lib/python3.6/site-packages/airflow/utils/session.py", line 70, in wrapper
  • return func(*args, session=session, **kwargs)
  • File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 592, in initdb
  • upgradedb(session=session)
  • File "/usr/local/lib/python3.6/site-packages/airflow/utils/session.py", line 67, in wrapper
  • return func(*args, **kwargs)
  • File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 920, in upgradedb
  • command.upgrade(config, 'heads')
  • File "/usr/local/lib/python3.6/site-packages/alembic/command.py", line 320, in upgrade
  • script.run_env()
  • File "/usr/local/lib/python3.6/site-packages/alembic/script/base.py", line 563, in run_env
  • util.load_python_file(self.dir, "env.py")
  • File "/usr/local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
  • module = load_module_py(module_id, path)
  • File "/usr/local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
  • spec.loader.exec_module(module) # type: ignore
  • File "", line 678, in exec_module
  • File "", line 219, in _call_with_frames_removed
  • File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py", line 107, in
  • run_migrations_online()
  • File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py", line 101, in run_migrations_online
  • context.run_migrations()
  • File "", line 8, in run_migrations
  • File "/usr/local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
  • self.get_context().run_migrations(**kw)
  • File "/usr/local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 620, in run_migrations
  • step.migration_fn(**kw)
  • File "/usr/local/lib/python3.6/site-packages/airflow/migrations/versions/a56c9515abdc_remove_dag_stat_table.py", line 39, in upgrade
  • op.drop_table("dag_stats")
  • File "", line 8, in drop_table
  • File "", line 3, in drop_table
  • File "/usr/local/lib/python3.6/site-packages/alembic/operations/ops.py", line 1349, in drop_table
  • operations.invoke(op)
  • File "/usr/local/lib/python3.6/site-packages/alembic/operations/base.py", line 392, in invoke
  • return fn(self, operation)
  • File "/usr/local/lib/python3.6/site-packages/alembic/operations/toimpl.py", line 80, in drop_table
  • operation.to_table(operations.migration_context)
  • File "/usr/local/lib/python3.6/site-packages/alembic/ddl/impl.py", line 372, in drop_table
  • self._exec(schema.DropTable(table))
  • File "/usr/local/lib/python3.6/site-packages/alembic/ddl/impl.py", line 193, in _exec
  • return conn.execute(construct, multiparams)
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
  • return meth(self, multiparams, params)
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
  • return connection._execute_ddl(self, multiparams, params)
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1073, in _execute_ddl
  • compiled,
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
  • e, statement, parameters, cursor, context
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
  • sqlalchemy_exception, with_traceback=exc_info[2], from_=e
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
  • raise exception
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
  • cursor, statement, parameters, context
  • File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
  • cursor.execute(statement, parameters)
  • sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) table "dag_stats" does not exist
  • [SQL:
  • DROP TABLE dag_stats]
  • (Background on this error at: http://sqlalche.me/e/13/f405)
  • I tried creating a table (dag_stats) in Postgres but still the issue continues.
subrat nayak
  • 21
  • 4

1 Answers1

0

You will have to upgrade the database, please refer this link for uprading Airflow version

S N
  • 540
  • 1
  • 6
  • 18