asyncio improperly warns about streams objects are garbage collected; call "stream.close()" explicitly

Question

I was implementing asynchronous MySQL query execution using python3.8's inbuilt asyncio package and an installed aiomysql package. Even though I have closed properly all the open cursor and connection, an error message keep on appearing on my console as follows.

An open stream object is being garbage collected; call "stream.close()" explicitly.

A summary of the code is given below...

#db.py

import asyncio

class AsyncMysqlSession:

    def __init__(self, loop, db_settings=DEFAULTDB):
        self.db_settings = db_settings
        self.loop = loop

    async def __aenter__(self):
        self.conn = await aiomysql.connect(host=self.db_settings['HOST'],
                                       port=self.db_settings['PORT'],
                                       user=self.db_settings['USER'],
                                       password=self.db_settings['PASSWORD'],
                                       db=self.db_settings['NAME'],
                                       loop=self.loop)
        self.cursor = await self.conn.cursor(aiomysql.cursors.DictCursor)
        return self

    async def __aexit__(self, exception, value, traceback):
        await self.cursor.close()
        self.conn.close()

    async def query(self, sql, *args):
        await self.cursor.execute(sql, values)
        await self.conn.commit()
        rows = await self.cursor.fetchall()
        return list(rows)


async def aiomysql_query(sql, *args):
    """
    Mysql asynchronous connection wrapper
    """
    loop = asyncio.get_event_loop()
    async with AsyncMysqlSession(loop) as mysql:
        db_result = await mysql.query(sql, *args)
        return db_result

aiomysql_query is imported in another file

#views.py

 import asyncio
 .....
 

 async def main():
     .....
     .....
     
     await aiomysql_query(sql1, *args1)
     await aiomysql_query(sql2, *args2)

 .....

 asyncio.run(main())

 ....

Am I doing something wrong here (?) or is it improperly shows the error message?. Any lead to resolve this issue will be appreciated... TIA!!

Just guessing. Could you try with added `await conn.wait_closed()` as recommended: https://docs.python.org/3/library/asyncio-stream.html#asyncio.StreamWriter.close ? — VPfB, Sep 02 '21 at 14:31

score 1 · Accepted Answer · answered Sep 06 '21 at 06:19

It seems like you may have just forgotten to close the event loop—in addition to await conn.wait_closed(), which @VPfB advised above.

You must close the event loop when manually using lower level method calls such as asyncio.get_event_loop(). Specifically, self.loop.close() must be called.

#db.py

import asyncio

class AsyncMysqlSession:

    def __init__(self, loop, db_settings=DEFAULTDB):
        self.db_settings = db_settings
        self.loop = loop

    async def __aenter__(self):
        self.conn = await aiomysql.connect(host=self.db_settings['HOST'],
                                       port=self.db_settings['PORT'],
                                       user=self.db_settings['USER'],
                                       password=self.db_settings['PASSWORD'],
                                       db=self.db_settings['NAME'],
                                       loop=self.loop)
        self.cursor = await self.conn.cursor(aiomysql.cursors.DictCursor)
        return self

    async def __aexit__(self, exception, value, traceback):
        await self.cursor.close()
        self.conn.close()
        self.loop.close()

    async def query(self, sql, *args):
        await self.cursor.execute(sql, values)
        await self.conn.commit()
        rows = await self.cursor.fetchall()
        return list(rows)


async def aiomysql_query(sql, *args):
    """
    Mysql asynchronous connection wrapper
    """
    loop = asyncio.get_event_loop()
    async with AsyncMysqlSession(loop) as mysql:
        db_result = await mysql.query(sql, *args)
        return db_result

References

https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.shutdown_asyncgens

https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.get_event_loop

asyncio improperly warns about streams objects are garbage collected; call "stream.close()" explicitly

1 Answers1

References