2

I've been scratching my head trying to figure out if this is possible.

I have a server program running with about 30 different socket connections to it from all over the country. I need to update this server program now and although the client devices will automatically reconnect, its not totally reliable.

I was wondering if there is a way of saving the socket object to a file? then load it back up when the server restarts? or forcefully keeping a socket open even after the program stops. This way the clients never disconnect at all.

Could really do with hot swappable code here really!

Harvey
  • 1,320
  • 3
  • 13
  • 33
  • 1
    What you're trying sounds a bit dangerous, I would rather look into ways of making sure reconnecting works 100% of the time. Why are the clients some times not able to reconnect? – Metareven May 02 '17 at 13:53

3 Answers3

3

Solution 1.

It can be done with some process magic, at least under linux (although I do believe similar windows api exists). First of all note that sockets cannot be stored in a file. These objects are temporary by their nature. But you can keep them in a separate process. Have a look at this:

Can I open a socket and pass it to another process in Linux

So one way to accomplish this is the following:

  1. Create a "keeper" process at some point (make sure that the process is not a child of the main process so that it stays alive when the main process is gone)
  2. Send all sockets to the keeper process via sendmsg() with SCM_RIGHTS
  3. Shutdown the main process
  4. Do whatever update you have to
  5. Fire the main process
  6. Retrieve sockets from the keeper process
  7. Shutdown the keeper process

However this solution is quite difficult to maintain. You have two separate processes, it is unclear which is the master and which is a slave. So you would probably need another master process at the top. Things get nasty very quickly, not to mention security issues.

Solution 2.

Reloading modules as suggested by @gavinb might be a solution. Note however that in practice this often breaks the app. You never know what those modules do under the hood unless you know the code of every single Python file you use. Plus it imposes some restrictions on modules, i.e. they have to be reloadable. For example some modules use inline caching which makes reloading difficult.

Also once a module is loaded in a different module it keeps a reference to that module. So you not only have to reload it but also update references in every other module that loaded it earlier. The maintanance costs raise very quickly unless you thought about it at the begining of the project (so that every import is encapsulated for easy reload). And bugs caused by two different versions of a module running in the same process are (I imagine, never been in this situation though) extremely difficult to find.

Anyway I would avoid that.

Solution 3.

So this is XY problem. Instead of saving sockets how about you put a proxy in front of the main server? IMO this is the safest and at the same time simpliest solution. The proxy will communicate with the main server (for example over unix domain sockets) and will buffer the data and automatically reconnect to the main server once it is available again. Perhaps you can even reuse some existing tech, e.g. nginx.

Community
  • 1
  • 1
freakish
  • 54,167
  • 9
  • 132
  • 169
2

No, the sockets are special file handles that belong to the process. If you close the process, the runtime will force close any open files/sockets. This is not Python specific; it is just how operating systems manage resources.

Now what you can do however is dynamically reload one or more modules while keeping the process active. It might take some careful management when you have open sockets, but in theory it should be possible. So yes, hot swappable code is actually supported by Python.

Do some reading and research on "dynamic reloading". The importlib module in Python 3 provides the reload function which is used to:

Reload a previously imported module. The argument must be a module object, so it must have been successfully imported before. This is useful if you have edited the module source file using an external editor and want to try out the new version without leaving the Python interpreter.

gavinb
  • 19,278
  • 3
  • 45
  • 60
  • `If you close the process, the runtime will force close any open files/sockets.` That's not true. You can share sockets between processes. Normally you do that with a child process, but you can even send it to some unrelated process: http://stackoverflow.com/questions/1997622/can-i-open-a-socket-and-pass-it-to-another-process-in-linux – freakish May 02 '17 at 14:00
  • @freakish That linked page relates to closing sockets from different threads within the same process, which is not the same situation as described here. Now the standard behaviour of `exit` is to close file descriptors, which includes sockets. https://superuser.com/questions/375604/does-windows-take-care-of-closing-sockets-when-processes-exit That behaviour does not preclude other mechanisms of explicitly sharing file descriptors (e.g. with `fork` or `sendmsg`). – gavinb May 02 '17 at 14:21
  • It is not talking about the same process, calling`sendmsg` to yourself is completely pointless - the process already has an access to its file descriptors. Perhaps you've read a wrong link I've posted earlier and fixed that. As for my comment: I'm only commenting on the highlighted sentence. It is false. I can close child process and sockets are still alive. As it is now your answer is suggesting something that is not true. And this is very important, because actually `sendmsg` opens one way for the OP to handle his problem. – freakish May 02 '17 at 14:26
0

I think your critical question is how to hot reload.

And as mentioned by @gavinb, you can import importlib and then use importlib.reload(module) to reload a module dynamically.

Be careful, the parameter of reload(param) must be a module.

Sraw
  • 18,892
  • 11
  • 54
  • 87