Python requests, request in del

Question

I am building an API client wrapper for some application. Therefore I created a Client class that will handle the requests. I was trying to send a logout request in the destructor, such that logout can not be forgotten if the program is ended.

Here is what I tried:

import requests

class Client:
    def __init__(self):
        # some stuff

    def login(self):
        requests.get(login_url_path, headers=some_headers)

    def logout(self):
        requests.post(logout_url_path, headers=some_headers)

    def __del__(self):
        self.logout()

if __name__ is '__main__':
    client = Client()
    client.login()

This results in:

File "/Users/me/Library/Python/3.9/lib/python/site-packages/requests/api.py", line 115, in post
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/requests/api.py", line 59, in request
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/requests/sessions.py", line 577, in request
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/requests/sessions.py", line 759, in merge_environment_settings
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/requests/utils.py", line 825, in get_environ_proxies
  File "/Users/me/Library/Python/3.9/lib/python/site-packages/requests/utils.py", line 809, in should_bypass_proxies
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 2647, in proxy_bypass
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 2624, in proxy_bypass_macosx_sysconf
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 2566, in _proxy_bypass_macosx_sysconf

ImportError: sys.meta_path is None, Python is likely shutting down

The problem appears to be that some imports no longer exit when the destructor is called as python is shutting down. Is this an expected issue? Is there a work around?

score 0 · Accepted Answer · answered Mar 10 '23 at 08:26

0

For details on the __del__() method, see the following question: What is the __del__ method and how do I call it?

In short, __del__() is called when an object instance is garbage collected. In your example this happens when your script comes to an end and the interpreter starts shutting down, at which point the latter's state is unpredictable.

If you want to make sure some cleanup operation takes place after the Client instance is no longer needed, a common approach is to add context manager capabilities to it, example:

class Client:
    ...
    def __enter__(self):  # Called upon entering the "with" block
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):  # Called when exiting the "with" block
        self.logout()

Then you can use a client instance like this:

with Client() as client_instance:
    client_instance.login()
    # and some more stuff using the client...

# After exiting the "with" block, we know that
# client_instance.__exit__() has already been called.

By using the client like this, you assure that logout() is automatically called when the script no longer needs it.

answered Mar 10 '23 at 08:26

plamut

3,085
10
29
40

Thanks a lot for the prompt reply! It looks like a clean and robust way to use the client indeed. However, this method will still rely on the person using the class to use a context manager. Also since the client will contain a DB that needs to be updated now and then, always opening up a context manager seems not super efficient. – SenneVP Mar 14 '23 at 11:16
That's true, the programmer still has _some_ responsibility, but at least they will not forget to cleanup after themselves. It's a common idiom in Python. Not sure about the _"containing a DB"_ part, can you elaborate? This sounds like an additional requirement that was not specified in the original answer. Guessing, maybe the `Client` class is designed to do too much, and adding a separate class for managing the communication with the database might be in order? – plamut Mar 14 '23 at 12:32
I mean the API that `Client` contacts is the API of an online database application. And thus someone using the `Client` will try to get some info at one point in time, then try to post some data at an other point, while complex operations might occure in between. `Client` is just the API wrapper, that will also manage the API token at the user end. – SenneVP Mar 14 '23 at 13:29
If I understand correctly, the `Client` is stateful? Would it make more sense to manage the "session state" on the server instead? The server would automatically close a "session" after a certain period of inactivity, in which case the `Client`would need to re-login, but an "active" client instance could still periodically send heartbeat request, informing the server to not close the "session" yet. Might be a better approach, especially since it's not guaranteed that the final `logout()` request will actually reach the server (network errors, `Client` unexpectedly crashing, etc.) – plamut Mar 15 '23 at 08:52
The API token provided by the server expires automatically after several hours. If the user logges out explicitly before that time, the token is deleted from the DB before it expires. So it is not an issue if the logout never happens. I just thought it was cleaner to remove the token after each "session", as they are probably much shorter than the expiration time. – SenneVP Mar 15 '23 at 09:13
This sounds good yes, having a timeout on the server, while also having a nicely behaving client to clean after itself early, when it knows it's done with all the work. With that said, and to return to the original question, a standard approach in Python is to add context manager capabilities to `Client`, even if the `Client` instance lifetime spans across the entire duration of the client application. Don't use `__del__()` for that, it has too many pitfalls and generally you cannot say for sure _when_ it will actually run. – plamut Mar 15 '23 at 09:24
Ok thanks! I will look into this option a bit more in that case – SenneVP Mar 15 '23 at 11:12

Python requests, request in __del__

1 Answers1

Python requests, request in del