1

Hi I want to use sqlalchemy to get connect to my table, get some column, do things with them and then move on. Problem is when i exit the function my ram space is not opened up. here is a dummy example.

from sqlalchemy import create_engine
import gc
mem = !free
dic = {}
dic_conn = {}
print(mem[1])
engine_ = create_engine(psql_string)
for tb in ['tb_1', 'tb_2', 'tb_3', 'tb_4',]:
    conn = engine_.connect()
    dic[tb] = conn.execute("select * from tb".format(tb))
    dic_conn[tb] = conn
for k in dic_conn.keys(): dic_conn[k].close(); dic_conn[k] = None
for k in dic.keys(): dic[k].close(); dic[k] = None
engine_.dispose()
del dic_conn
del dic
del engine_
gc.collect()
mem = !free
print(mem[1])

i tried anything I could think of, let me know how I can free up that memory

snakecharmerb
  • 47,570
  • 11
  • 100
  • 153
user1871528
  • 1,655
  • 3
  • 27
  • 41
  • Does this answer your question? [Releasing memory in Python](https://stackoverflow.com/questions/15455048/releasing-memory-in-python) – snakecharmerb Dec 17 '21 at 07:13
  • No its not because its not a python thing, its appears to be a sql alchemy thing. sqlalchemy doesnt release that memory – user1871528 Dec 17 '21 at 18:21
  • The connection `conn` is never closed, so that could potentially leave a postgres connection open, which would show as a Python process. Other than that, is isn't very clear what exactly your code is demonstrating.`mem = !free` is not valid syntax. Where do you see SQLAlchemy retaining RAM? How do you know it is SQLAlchemy? – snakecharmerb Dec 17 '21 at 18:50
  • 1
    !free is a jupyter cheatcode – user1871528 Dec 18 '21 at 07:40

1 Answers1

0

sqlalchemy holds the buffer side on the client side, you can alternatively have the buffer on server side and that way sqlalchemy frees the buffer space. this command will do it.

conn = engine_.execution_options(stream_results=True).connect()
user1871528
  • 1,655
  • 3
  • 27
  • 41