Python3: Wait for Daemon to finish iteration

Question

I'm writing a python script that will start a local fileserver, and while that server is alive it will be writing to a file every 30 seconds. I would like to have the server and writer function running synchronously so I made the writer function a daemon thread... My main question is, since this daemon thread will quit once the server is stopped, if the daemon is in the middle of writing to a file will it complete that operation before exiting? It would be really bad to be left with 1/2 a file. Here's the code, but the actual file it will be writing is about 3k lines of JSON, hence the concern.

import http.server
import socketserver
from time import sleep
from threading import Thread


class Server:
    def __init__(self):
        self.PORT = 8000
        self.Handler = http.server.SimpleHTTPRequestHandler
        self.httpd = socketserver.TCPServer(("", self.PORT), self.Handler)
        print("Serving at port", self.PORT)

    def run(self):
        try:
            self.httpd.serve_forever()
        except KeyboardInterrupt:
            print("Server stopped")


def test():
    while True:
        with open('test', mode='w') as file:
            file.write('testing...')
        print('file updated')
        sleep(5)


if __name__ == "__main__":
    t = Thread(target=test, daemon=True)
    t.start()
    server = Server()
    server.run()

score 3 · Accepted Answer · edited May 23 '17 at 12:10

It looks like you may have made an incorrect decision making the writer thread daemonic.
Making a daemonic thread does not mean it will run synchronously. It will still be affected by the GIL.
If you want synchronous execution, you'll have to use multiprocessing

From the Python docs:

Daemon threads are abruptly stopped at shutdown. Their resources (such as open files, database transactions, etc.) may not be released properly. If you want your threads to stop gracefully, make them non-daemonic and use a suitable signalling mechanism such as an Event.

So that means that daemon threads are only suitable for the tasks that only make sense in context of the main thread and don't matter when the main thread has stopped working. File I/O, particularly data saving, is not suitable for a daemon thread.

So it looks like the most obvious and logical solution would be to make the writer thread non-daemonic.
Then, even if the main thread exits, the Python process won't be ended until all non-daemonic threads have finished. This allows for file I/O to complete and exit safely.

Explanation of daemonic threads in Python can be found here

Python3: Wait for Daemon to finish iteration

1 Answers1