Airflow initdb slot_pool does not exists

Question

I'm facing an issue with airflow initialization on postgres backend

Ubuntu : 18.04.1

Airflow : v1.10.6

Postgres : 10.10

Python 3.6

And when I run

airflow initdb

I get

    [2019-11-22 10:17:23,564] {db.py:368} INFO - Creating tables
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedTable: relation "airflow.slot_pool" does not exist
LINE 2: FROM airflow.slot_pool
             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 37, in <module>
    args.func(args)
  File "/usr/local/lib/python3.6/dist-packages/airflow/bin/cli.py", line 1131, in initdb
    db.initdb(settings.RBAC)
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 106, in initdb
    upgradedb()
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 377, in upgradedb
    add_default_pool_if_not_exists()
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 74, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 90, in add_default_pool_if_not_exists
    if not Pool.get_pool(Pool.DEFAULT_POOL_NAME, session=session):
  File "/usr/local/lib/python3.6/dist-packages/airflow/utils/db.py", line 70, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/airflow/models/pool.py", line 44, in get_pool
    return session.query(Pool).filter(Pool.pool == pool_name).first()
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 3265, in first
    ret = list(self[0:1])
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 3043, in __getitem__
    return list(res)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 3367, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 982, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
    distilled_params,
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
    e, statement, parameters, cursor, context
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "airflow.slot_pool" does not exist
LINE 2: FROM airflow.slot_pool
             ^

[SQL: SELECT airflow.slot_pool.id AS airflow_slot_pool_id, airflow.slot_pool.pool AS airflow_slot_pool_pool, airflow.slot_pool.slots AS airflow_slot_pool_slots, airflow.slot_pool.description AS airflow_slot_pool_description
FROM airflow.slot_pool
WHERE airflow.slot_pool.pool = %(pool_1)s
 LIMIT %(param_1)s]
[parameters: {'pool_1': 'default_pool', 'param_1': 1}]
(Background on this error at: http://sqlalche.me/e/f405)

I've tried deleting / recreating database and user rights (with search_path as said in doc). My postgres is accessible and well configured as tables have been previously created by airflow before the crash ;)

Any ideas?

I've made a try with with Airflow 1.10.2 and it work smoothly with postgres backend.

I've downgrade my Airflow version to 1.10.2. – airliquide Dec 19 '19 at 08:25 — airliquide, Dec 19 '19 at 08:25

score 4 · Answer 1 · answered Feb 09 '20 at 13:03

4

This is might be because of examples. Try load_examples = False in airflow.cfg andrun airflow upgradedb or airflow resetdb

answered Feb 09 '20 at 13:03

VirtualLogic

706
1
10
25

score 2 · Answer 2 · edited Jun 20 '20 at 09:12

2

Have you tried

airflow resetdb

airflow initdb: Initialize the metadata database

airflow resetdb: Burn down and rebuild the metadata database

airflow upgradedb: Apply missing migrations - this is also idempotent and safe but it tracks migrations so if your tables aren't in the state that alembic thinks that they are in you would need to find that "state" and edit that

Most importantly, if you have any connections or variables set they will be deleted if you run this.

edited Jun 20 '20 at 09:12

Community

1
1

answered Jan 02 '20 at 09:00

Daniel Lee

7,189
2
26
44

this just gives me a different error: `psycopg2.errors.DuplicateColumn: column "operator" of relation "task_instance" already exists` – Brad Jan 20 '20 at 03:53
i have this same error too. I happened when I downgraded back to 1.10.7 after an upgrade to 1.10.10. How did you fix yours without resetdb? – alltej Apr 24 '20 at 13:29
You could try running `airflow upgradedb` – Daniel Lee Apr 24 '20 at 15:46

score 1 · Answer 3 · answered Oct 05 '20 at 08:51

1

Airflow uses default backend SQLite, and have default tables(like slot_pool) there. So if you consider to use another backend like postgres you should at least add these default tables there.

$ airflow initdb

answered Oct 05 '20 at 08:51

Raymond Reddington

1,709
1
13
21

score 1 · Answer 4 · answered Aug 29 '21 at 13:04

I know it's kinda late, but I experienced the same trying $ airflow db init against a MySQL 8 backend, and got a similar error.

Turned out that there was a mismatch in my configuration: The database was named airflow_db, so I had sql_alchemy_conn = mysql+mysqlconnector://airflow:******@localhost:3306/airflow_db, but a few lines down in the airflow.cfg, I also had the line sql_alchemy_schema = airflow

Airflow initdb slot_pool does not exists

4 Answers4

Linked