1

I'm trying add a user to a PSQL DB using SQL Alchemy and docker-compose. I have 2 containers: 'db' and 'py'. After building and running the containers, I hit the API in the 'py' container to store the user. When I run db_session.commit() to save the user I get the following error:

RecursionError: maximum recursion depth exceeded

This code works perfectly when I run it locally. I change the DB host to localhost and the user is correctly stored in the DB. When run in the docker containers, I get the recursion error.

db/handler.py

    @contextmanager
    def session_scope():
    engine = create_engine('postgresql+psycopg2://postgres:postgres@db:5432/my_db', echo=True)
    session_factory = sessionmaker(bind=engine)
    db_session = scoped_session(session_factory)
    try:
        yield db_session
    except:
        db_session.rollback()
    finally:
        db_session.close()

docker-compose.yaml

services:
    py01:
        build: ./py
        image: py
        depends_on:
            - db
        container_name: py
        expose:
            - 8000

    db01:
        build: ./db
        image: db
        container_name: db
        ports:
            - "5432:5432"

users.py

with session_scope() as db_session:
    user = User(user_class="tempweb", temp_web_id=temp_web_id)

    db_session.add(user)

    db_session.commit() # <---- MAXIMUM RECURSION DEPTH EXCEEDED error occurs here

Stack traceback

py         | ERROR:    Exception in ASGI application
py         | Traceback (most recent call last):
py         |   File "/opt/pypy/site-packages/uvicorn/protocols/http/h11_impl.py", line 388, in run_asgi
py         |     result = await app(self.scope, self.receive, self.send)
py         |   File "/opt/pypy/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
py         |     return await self.app(scope, receive, send)
py         |   File "/opt/pypy/site-packages/fastapi/applications.py", line 182, in __call__
py         |     await super().__call__(scope, receive, send)  # pragma: no cover
py         |   File "/opt/pypy/site-packages/starlette/applications.py", line 111, in __call__
py         |     await self.middleware_stack(scope, receive, send)
py         |   File "/opt/pypy/site-packages/starlette/middleware/errors.py", line 181, in __call__
py         |     raise exc from None
py         |   File "/opt/pypy/site-packages/starlette/middleware/errors.py", line 159, in __call__
py         |     await self.app(scope, receive, _send)
py         |   File "/opt/pypy/site-packages/starlette/middleware/cors.py", line 78, in __call__
py         |     await self.app(scope, receive, send)
py         |   File "/opt/pypy/site-packages/starlette/exceptions.py", line 82, in __call__
py         |     raise exc from None
py         |   File "/opt/pypy/site-packages/starlette/exceptions.py", line 71, in __call__
py         |     await self.app(scope, receive, sender)
py         |   File "/opt/pypy/site-packages/starlette/routing.py", line 566, in __call__
py         |     await route.handle(scope, receive, send)
py         |   File "/opt/pypy/site-packages/starlette/routing.py", line 227, in handle
py         |     await self.app(scope, receive, send)
py         |   File "/opt/pypy/site-packages/starlette/routing.py", line 41, in app
py         |     response = await func(request)
py         |   File "/opt/pypy/site-packages/fastapi/routing.py", line 197, in app
py         |     dependant=dependant, values=values, is_coroutine=is_coroutine
py         |   File "/opt/pypy/site-packages/fastapi/routing.py", line 149, in run_endpoint_function
py         |     return await run_in_threadpool(dependant.call, **values)
py         |   File "/opt/pypy/site-packages/starlette/concurrency.py", line 34, in run_in_threadpool
py         |     return await loop.run_in_executor(None, func, *args)
py         |   File "/opt/pypy/lib-python/3/asyncio/futures.py", line 327, in __iter__
py         |     yield self  # This tells Task to wait for completion.
py         |   File "/opt/pypy/lib-python/3/asyncio/tasks.py", line 250, in _wakeup
py         |     future.result()
py         |   File "/opt/pypy/lib-python/3/asyncio/futures.py", line 243, in result
py         |     raise self._exception
py         |   File "/opt/pypy/lib-python/3/concurrent/futures/thread.py", line 56, in run
py         |     result = self.fn(*self.args, **self.kwargs)
py         |   File "./app/routers/users.py", line 33, in get_session
py         |     with session_scope() as db_session:
py         |   File "/opt/pypy/lib-python/3/contextlib.py", line 81, in __enter__
py         |     return next(self.gen)
py         |   File "./app/db/handler.py", line 10, in session_scope
py         |     engine.connect()
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/base.py", line 2265, in connect
py         |     return self._connection_cls(self, **kwargs)
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/base.py", line 104, in __init__
py         |     else engine.raw_connection()
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/base.py", line 2372, in raw_connection
py         |     self.pool.unique_connection, _connection
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/base.py", line 2338, in _wrap_pool_connect
py         |     return fn()
py         |   File "/opt/pypy/site-packages/sqlalchemy/pool/base.py", line 304, in unique_connection
py         |     return _ConnectionFairy._checkout(self)
py         |   File "/opt/pypy/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
py         |     fairy = _ConnectionRecord.checkout(pool)
py         |   File "/opt/pypy/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
py         |     rec = pool._do_get()
py         |   File "/opt/pypy/site-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
py         |     self._dec_overflow()
py         |   File "/opt/pypy/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
py         |     exc_value, with_traceback=exc_tb,
py         |   File "/opt/pypy/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
py         |     raise exception
py         |   File "/opt/pypy/site-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
py         |     return self._create_connection()
py         |   File "/opt/pypy/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
py         |     return _ConnectionRecord(self)
py         |   File "/opt/pypy/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
py         |     self.__connect(first_connect_check=True)
py         |   File "/opt/pypy/site-packages/sqlalchemy/pool/base.py", line 666, in __connect
py         |     ).exec_once_unless_exception(self.connection, self)
py         |   File "/opt/pypy/site-packages/sqlalchemy/event/attr.py", line 314, in exec_once_unless_exception
py         |     self._exec_once_impl(True, *args, **kw)
py         |   File "/opt/pypy/site-packages/sqlalchemy/event/attr.py", line 285, in _exec_once_impl
py         |     self(*args, **kw)
py         |   File "/opt/pypy/site-packages/sqlalchemy/event/attr.py", line 322, in __call__
py         |     fn(*args, **kw)
py         |   File "/opt/pypy/site-packages/sqlalchemy/util/langhelpers.py", line 1513, in go
py         |     return once_fn(*arg, **kw)
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/strategies.py", line 199, in first_connect
py         |     dialect.initialize(c)
py         |   File "/opt/pypy/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 732, in initialize
py         |     super(PGDialect_psycopg2, self).initialize(connection)
py         |   File "/opt/pypy/site-packages/sqlalchemy/dialects/postgresql/base.py", line 2562, in initialize
py         |     super(PGDialect, self).initialize(connection)
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/default.py", line 313, in initialize
py         |     connection
py         |   File "/opt/pypy/site-packages/sqlalchemy/dialects/postgresql/base.py", line 2803, in _get_server_version_info
py         |     v = connection.execute("select version()").scalar()
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/base.py", line 1003, in execute
py         |     return self._execute_text(object_, multiparams, params)
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/base.py", line 1178, in _execute_text
py         |     parameters,
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
py         |     e, statement, parameters, cursor, context
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/base.py", line 1514, in _handle_dbapi_exception
py         |     util.raise_(exc_info[1], with_traceback=exc_info[2])
py         |   File "/opt/pypy/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
py         |     raise exception
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/base.py", line 1294, in _execute_context
py         |     result = context._setup_crud_result_proxy()
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/default.py", line 1243, in _setup_crud_result_proxy
py         |     result = self.get_result_proxy()
py         |   File "/opt/pypy/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 601, in get_result_proxy
py         |     return _result.ResultProxy(self)
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/result.py", line 775, in __init__
py         |     self._init_metadata()
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/result.py", line 807, in _init_metadata
py         |     self._metadata = ResultMetaData(self, cursor_description)
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/result.py", line 296, in __init__
py         |     textual_ordered,
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/result.py", line 514, in _merge_cursor_description
py         |     ) in raw_iterator
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/result.py", line 497, in <listcomp>
py         |     (
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/result.py", line 621, in _merge_cols_by_none
py         |     ) in self._colnames_from_description(context, cursor_description):
py         |   File "/opt/pypy/site-packages/sqlalchemy/engine/result.py", line 541, in _colnames_from_description
py         |     colname = rec[0]
py         | RecursionError: maximum recursion depth exceeded
marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Lee Mordell
  • 473
  • 5
  • 16

1 Answers1

1

I figured out the issue. It was caused by using the psycopg2 driver in the DB connection string. When I went into the python interpreter and tried to import psycopg2, I got an import error. There was an issue with the python installed on the Docker container (I used pypy:3) that caused this error to occur in the container, but not locally.

ImportError: No module named 'psycopg2._psycopg'

To fix, I just chose a different dialect in the DB connection string: pg8000 and it now works perfectly in the container.

Thanks to this SO post for helping to diagnose the issue.

Lee Mordell
  • 473
  • 5
  • 16
  • Also, just to be curious, why are you using `postgresql+psycopg2` in your driver part of the conn string. I am assuming you have built psycopg from source? – onlinejudge95 Sep 18 '20 at 05:22
  • 1
    The SQL Alchemy docs used ```postgresql+psycopg2``` in the tutorial as part of the DB connection URI. I included it in requirements.txt to add to the project. Like I said, it worked fine locally, but I think the python build in the docker container was missing some libraries ```psycopg2``` needed to run, thus the module import error. – Lee Mordell Sep 18 '20 at 14:56