1

I do not want to lose my sets if windows is about to shutdown/restart/log off/sleep, Is it possible to save it before shutdown? Or is there an alternative to save information without worring it will get lost on windows shutdown? JSON, CSV, DB? Anything?

s = {1,2,3,4}

with open("s.pick","wb") as f: # pickle it to file when PC about to shutdown to save information
    pickle.dump(s,f)
Maarten Bodewes
  • 90,524
  • 13
  • 150
  • 263
  • 1
    What's the problem with using `pickle`? – jonrsharpe Aug 03 '14 at 11:20
  • 1
    How important is it that this data is saved? What happens if the machine hangs or abruptly loses power? – user2357112 Aug 03 '14 at 11:29
  • @user2357112: Good point. Design to handle arbitrary termination first, then add in code that takes advantage of shutdown messaging to handle that case even better. – abarnert Aug 03 '14 at 21:50
  • @abarnert it's possible to handle arbitrary termination? If I have function which is running in for loop in try catch (inside expetion I will do with open?) this will work?\ –  Aug 06 '14 at 18:56
  • @IlanTiger: There are cases that you can't possibly detect and handle, like someone pulling the power, or the CPU catching fire. So you need to design your code so that, no matter how it gets shut down, you will be able to recover. The basic idea is to never modify any critical external data without some kind of transaction with atomic commit. That could just mean atomic-write-temp-and-replace for a single file, or `begin transaction` for a database, or it could mean all kinds of journaling to keep track of dependencies between files or distributed systems, depending on your app. – abarnert Aug 07 '14 at 23:13

2 Answers2

5

I do not want to lose my sets if windows is about to shutdown/restart/log off/sleep, Is it possible to save it before shutdown?

Yes, if you've built an app with a message loop, you can receive the WM_QUERYENDSESSION message. If you want to have a GUI, most GUI libraries will probably wrap this up in their own way. If you don't need a GUI, your simplest solution is probably to use PyWin32. Somewhere in the docs there's a tutorial on creating a hidden window and writing a simple message loop. Just do that on the main thread, and do your real work on a background thread, and signal your background thread when a WM_QUERYENDSESSION message comes in.

Or, much more simply, as Evgeny Prokurat suggests, just use SetConsoleCtrlHandler (again through PyWin32). This can also catch ^C, ^BREAK, and the user closing your console, as well as the logoff and shutdown messages that WM_QUERYENDSESSION catches. More importantly, it doesn't require a message loop, so if you don't have any other need for one, it's a lot simpler.


Or is there an alternative to save information without worring it will get lost on windows shutdown? JSON, CSV, DB? Anything?

The file format isn't going to magically solve anything. However, a database could have two advantages.

First, you can reduce the problem by writing as often as possible. But with most file formats, that means rewriting the whole file as often as possible, which will be very slow. The solution is to streaming to a simpler "journal" file, packing that into the real file less often, and looking for a leftover journal at every launch. You can do that manually, but a database will usually do that for you automatically.

Second, if you get killed in the middle of a write, you end up with half a file. You can solve that by the atomic writing trick—write a temporary file, then replace the old file with the temporary—but this is hard to get right on Windows (especially with Python 2.x) (see Getting atomic writes right), and again, a database will usually do it for you.


The "right" way to do this is to create a new window class with a msgproc that dispatches to your handler on WM_QUERYENDSESSION. Just as MFC makes this easier than raw Win32 API code, win32ui (which wraps MFC) makes this easier than win32api/win32gui (which wraps raw Win32 API). And you can find lots of samples for that (e.g., a quick search for "pywin32 msgproc example" turned up examples like this, and searches for "python win32ui" and similar terms worked just as well).

However, in this case, you don't have a window that you want to act like a normal window, so it may be easier to go right to the low level and write a quick&dirty message loop. Unfortunately, that's a lot harder to find sample code for—you basically have to search the native APIs for C sample code (like Creating a Message Loop at MSDN), then figure out how to translate that to Python with the pywin32 documentation. Less than ideal, especially if you don't know C, but not that hard. Here's an example to get you started:

def msgloop():
    while True:
        msg = win32gui.GetMessage(None, 0, 0)
        if msg and msg.message == win32con.WM_QUERYENDSESSION:
            handle_shutdown()
        win32api.TranslateMessage(msg)
        win32api.DispatchMessage(msg)
        if msg and msg.message == win32con.WM_QUIT:
            return msg.wparam

worker = threading.Thread(real_program)
worker.start()
exitcode = msgloop()
worker.join()
sys.exit(exitcode)

I haven't shown the "how to create a minimal hidden window" part, or how to signal the worker to stop with, e.g., a threading.Condition, because there are a lot more (and easier-to-find) good samples for those parts; this is the tricky part to find.

abarnert
  • 354,177
  • 51
  • 601
  • 671
  • I think that guide for hidden console will be very useful any link for it? I have tried looking for it `SetConsoleCtrlHandler` will also need a hidden console window for it to work – Boris Daka Aug 03 '14 at 12:53
  • @BorisDaka: When a normal `.py` file is double-clicked, it gets a console to run it; when a user runs a `.py` file from the command line, it inherits the command line's console; etc. It's only when you use a `.pyw` file that you don't get a console—but the only reason to use a `.pyw` file is that you want to have a GUI, in which case you can use the other mechanism. – abarnert Aug 03 '14 at 13:44
  • I'm using cx-freeze windows .exe file does this make a difference? – Boris Daka Aug 03 '14 at 13:47
  • @BorisDaka: I'm pretty sure `cx_Freeze` can be used for both console and GUI apps. I can't remember if you have to do anything different in the setup file, or if it just depends on whether your main script is a `.py` or `.pyw`, but either way, you should be able to figure out which one you're building and do the appropriate thing for that case (or switch to the other case, if you discover it's not what you wanted). – abarnert Aug 03 '14 at 21:49
  • @abarnert hey, I have researched and tested when console app `SetConsoleCtrlHandler` works great, but when It's gui it's not the case (yes there is GUI functions of exit) but I need this right before the GUI and I don't show a console. Is there a way in that scenario I should use WM_QUERYENDSESSION do you have that page where they show that example I have no success recreating it –  Aug 06 '14 at 15:26
  • @IlanTiger: I don't know that there's any good sample code for implementing a `WM_QUERYENDSESSION` handler specifically. I'm sure there are good examples of writing a message loop in general, but… I don't actually know where one is, and I think it'll be quicker to edit one into the answer than to find one, so I'll do that. – abarnert Aug 06 '14 at 16:24
  • @abarnert thank you I have actually did find a working script to create a hidden window and it works to receive `WM_QUERYENDSESSION` [HERE](http://stackoverflow.com/questions/1411186/python-windows-shutdown-events/1411322) now the `win32gui.PumpWaitingMessages()` is the real program it will loop endlessy (and block) and I will need use your idea with the background thread I saw your code here I will try put up something but I have no experience with threading but thank you i'm very close now :) –  Aug 06 '14 at 17:13
  • @IlanTiger: That one does it the "right" way instead of the simple way, so it's a bit more code… but that's fine, better to learn the right way anyway. Anyway, the threading code still works exactly the same way I showed; just use `while True: win32gui.PumpWaitingMessages()` instead of `msgloop()`. If you need help signaling one thread from another, and searching for examples on `threading.Condition` doesn't help, you should probably write a new question on that, because it's a completely different problem, and people who know nothing about `pywin32` will be able to help you with it. – abarnert Aug 06 '14 at 17:20
  • @abarnert here is how i got it so far, [paste](http://pastebin.com/8Rb0duM0) you can see where i want to save my list if there is signal can this be done? –  Aug 06 '14 at 17:58
  • @IlanTiger: You have to actually call `checkforsignal` somewhere. And, since it never returns, you have to first spin off that `for` loop to run in another thread (the `real_program` function in my example above). And you need some way for the GUI thread to signal the worker thread. The simplest is probably to share a global `queue.Queue`; in the worker, do a non-blocking `get` or empty check each time through the loop, or maybe every N times, and exit early if you find something; then, in the GUI, in your `WM_QUERYENDSESSION` handler, just do a `put`. – abarnert Aug 07 '14 at 23:16
4

you can detect windows shutdown/log off with win32api.setConsoleCtrlHandler

there is a good example How To Catch “Kill” Events with Python

Evgeny Prokurat
  • 704
  • 5
  • 8
  • Nice, I completely forgot about that. – abarnert Aug 03 '14 at 11:26
  • @Evgeny Sadly this only works fine if you run a script on a console with python.exe, but not if you run it in the background with pythonw.exe. – Boris Daka Aug 03 '14 at 12:50
  • 1
    @BorisDaka: Then don't run it in the background; run it with a hidden console. If you really want `pythonw`, you can create a hidden user window and use the other mechanism in my answer. – abarnert Aug 03 '14 at 13:46