Python: How to ensure that class instance cleans up resources

Question

First let me acknowledge that a good discussion already exists here: How do I correctly clean up a Python object?

I'm trying to improve efficiency of a bit of code that currently is schematically like this:

import psycopg2

class StuffHandler:
    def __init__(self, db_credentials):
        self.db_credentials = db_credentials
    def handle(self, stuff):
        try:
            with psycopg2.connect(self.db_credentials) as conn:
                with conn.cursor() as cursor:
                    cursor.execute( < write stuff to database > )
        except psycopg2.DataError as e:
            logger.error(e)
        finally:
            conn.close()

As you see, this is correct, but it unnecessarily open and closes a database connection every time, which I'm trying to improve.

The code above is used by the application in the following way, schematically:

h = StuffHandler(db_credentials)
my_obj.set_stuff_handler(h)

I want this object to hold a long-lived connection, and schematically I think it should work like this

import psycopg2

class Handler:
    def __init__(self, db_credentials):
        self.conn = psycopg2.connect(db_credentials)
    def handle(self, stuff):
        # TODO: check that connection is still alive, but those are
        # details don't matter for this discussion
        try:
            with self.conn.cursor() as cursor:
                cursor.execute( < write stuff to database > )
        except psycopg2.DataError as e:
            logger.error(e)

The question is: in Python, how do I correctly clean up resources? That includes closing the connection, but also things like if I want to add a buffer, flush the buffer, etc.

According to the marked-as-correct and vastly more updated answer, https://stackoverflow.com/a/865272/12595391, the answer is: add methods __enter__ and __exit__ so that this class can be used as a context. I imagine that this means:

def __enter__(self):
    return self
def __exit__(exc_type, exc_value, traceback):
    self.conn.close()

So now my question becomes: what if I don't want to use contexts? What if I DON'T want to do

with StuffHandler(db_credentials) as h:
    my_obj.set_stuff_handler(h)

but instead I don't want to touch the application code?

In the question mentioned above, an alternative is described in this answer: https://stackoverflow.com/a/41627098/12595391 where they suggest registering a cleanup methods using atexit.register(). Isn't this vastly more appropriate for my situation? What are the downsides of this approach? The commenter @hlongmore seems to mention a downside but I don't really understand what he meant.

Thanks for the help.

The described downside is that, since you are registering an instance for exit, that instance is forever off-limits for garbage collection. If you only have one instance, that's fine. If you are making them dynamically (and expect all of them to be cleaned up at exit), you might exhaust your resources if your application is long-lived, or creates such objects rapidly. With that, is there anything left unclear, given the two links you provided? — Amadan, Dec 25 '19 at 11:58
Yes: So how can I correctly handle the all four cases: A) program exits, B) object is garbage collected, C) object's `__del__` is called, D) interpreter crashes? without contexts? — jijida, Dec 25 '19 at 12:02
The way that does all that (except interpreter crash, that scenario has no solution) involves `weakref.finalize`. — Amadan, Dec 25 '19 at 12:11

score 1 · Answer 1 · answered Dec 25 '19 at 12:03

You can try to use __del__, thought context manager would be preferable

import psycopg2

class Handler:
    def __init__(self, db_credentials):
        self.conn = psycopg2.connect(db_credentials)
    def handle(self, stuff):
        # TODO: check that connection is still alive, but those are
        # details don't matter for this discussion
        try:
            with self.conn.cursor() as cursor:
                cursor.execute( < write stuff to database > )
        except psycopg2.DataError as e:
            logger.error(e)
    def __del__():
        self.conn.close()

Its called when an object is garbage collected, What is the __del__ method, How to call it?

Thanks for responding, but the top 3 answers in the link you provided explicitly advises against this: e.g. "Since you have no guarantee it's executed, one should never put the code that you need to be run into __del__() — instead, this code belongs to finally clause of the try block or to a context manager in a with statement." Again, that person is suggesting contexts, but I don't want to have to change the application code. — jijida, Dec 25 '19 at 12:08

Python: How to ensure that class instance cleans up resources

1 Answers1