0

I'm trying to create a background service in Python. The service will be called from another Python program. It needs to run as a daemon process because it uses a heavy object (300MB) that has to be loaded previously into the memory. I've had a look at python-daemon and still haven't found out how to do it. In particular, I know how to make a daemon run and periodically do some stuff itself, but I don't know how to make it callable from another program. Could you please give some help?

martineau
  • 119,623
  • 25
  • 170
  • 301
lenhhoxung
  • 2,530
  • 2
  • 30
  • 61
  • @mithatkonuk I would prefer Python daemon to system daemon as suggested in this answer: http://stackoverflow.com/questions/1603109/how-to-make-a-python-script-run-like-a-service-or-daemon-in-linux. Is there any way to make a Python daemon callable, or it is mandatory to use a system daemon? – lenhhoxung Aug 07 '16 at 08:38

1 Answers1

1

I had a similar situation when I wanted to access a big binary matrix from a web app.

Of course there are many solutions, but I used Redis, a popular in-memory database/cache system, to store and access my object successfully. It has practical Python bindings (several probably equivalent wrapper libraries).

The main advantage is that when the service goes down, a copy of the data still remains on disk. Also, I noticed that once in place, it could be used for other things in my app (for instance Celery proposes it as backend), and actually, for other services in any other unrelated program.

JulienD
  • 7,102
  • 9
  • 50
  • 84
  • Can Redis be used with arbitrary objects? In fact, my object is a math model from tensorFlow – lenhhoxung Aug 07 '16 at 14:01
  • Not arbitrary. It can be lists, hashes, etc., see http://redis.io/topics/data-types. But for a Python object you will be forced to pickle it, then store as a string, and deserialize when you need it. You can also store the object attributes separately if they are simple data types. – JulienD Aug 07 '16 at 21:26