Do I need to implement the Dispose Pattern for a resource wrapper in Python

Question

If I am going to implement a safe resource wrapper in Python, do I need to implement the Dispose Pattern like C#?

Here is a demo implementation of what I mean:

class ResourceWrapper:
    def __init__(self):
        self._python_resource = ...  # A Python object that manages some resources.
        self._external_resource = _allocate_resource()  # A resource handle to an external resource.
        self._is_closed = False  # Whether the object has been closed.

    def __del__(self):
        self._close(manual_close=False)  # Called by GC.

    def close(self):
        self._close(manual_close=True)  # Called by user to free resource early.

    def _close(self, manual_close):
        if not self._is_closed:  # Don’t want a resource to be closed more than once.
            if manual_close:
                # Since `_close` is called by user, we can guarantee that `self._python_resource` is still valid, so we
                # can close it safely.
                self._python_resource.close() 
            else:
                # This means `_close` is called by GC, `self._python_resource` might be already GCed, but we don’t know
                # for sure, so we do nothing and rely on GC to free `self._python_resource`.

                pass

            # GC will not take care of freeing unmanaged resource, so whether manual close or not, we have to close the
            # resource to prevent leaking.

            _free_resource(self._external_resource)

            # Now we mark the object as closed to prevent closing multiple times.

            self._is_closed = True

self._python_resource is a resource wrapper object managed by Python GC, and self._external_resource is a handle to an external resource that is not managed by Python GC.

I want to ensure both managed and unmanaged resource gets freed if user manual closes the wrapper, and, they also gets freed if the wrapper object gets GCed.

el.pescado - нет войне · Accepted Answer · 2018-11-02T14:48:35.177

No, in Python you should use Context Managers:

class ResourceWrapper:
    def __init__(self):
        ...

    ...


    def __enter__(self):
        return self

    def __exit__(self, type, value, traceback):
        self._close(manual_close=False)

with ResourceWrapper() as wrapper:
    # do something with wrapper

Note 1: There's this comment in _close() method:

This means _close is called by GC, self._python_resource might be already GCed, but we don’t knowfor sure, so we do nothing and rely on GC to free self._python_resource.

I'm not sure what you mean by that, but as long as you hold reference to an object (and as long as it isn't a weak reference) it won't be GC'ed.

Note 2: What happens if an object that is a context manager is used without with block? Then resource will be released when object is garbage collected - but I wouldn't worry about that. Using context managers is common idiom in python (see any example with open()ing file). If that's crucial for your application, you can acquire resources in __enter__(), that way won't be acquired unless in with block.

Note 3, about cyclic references: If you have two objects that hold reference to each other, you've formed cyclic reference, so that two object won't be freed by "regular" reference-counting GC. Instead, they are to be collected by generational GC, unless thay happen to have __del__ method. __del__ inhibits GC from collecting objects. See gc.garbage:

A list of objects which the collector found to be unreachable but could not be freed (uncollectable objects). By default, this list contains only objects with __del__() methods. [1] Objects that have __del__() methods and are part of a reference cycle cause the entire reference cycle to be uncollectable, including objects not necessarily in the cycle but reachable only from it.

Python 3.4 introduced PEP-442, which introduces safe object finalization. Either way, you won't have invalid references. If you have attribute (hasattr(self, "_python_resource")) it will be valid.

Takeaway: don't use __del__.

What if user created a wrapper, but forgot to use it in a `with` block, wouldn't the resource be leaked? — EFanZh, Nov 02 '18 at 14:18
If there are two objects holding each other’s reference (strong reference), then when the two object gets GCed, one object must be freed before the other one, so when the second object gets GCed, the reference to the first one becomes invalid. — EFanZh, Nov 02 '18 at 14:22
About **Note 2**: there are two kinds of resources. One type is normal python objects managed by GC, Another type is a handle type (maybe a pointer or integer) that points to an external resource that can not be managed by GC, we have to manually free it using certain function calls. — EFanZh, Nov 02 '18 at 14:34

Do I need to implement the Dispose Pattern for a resource wrapper in Python

1 Answers1