Python, Django and event loop (periodic jobs)

Question

I am developing a Python applicaton for server that uses Django + WSGI + Apache under Debian Linux. The application has web interface as well as command line interface (that still uses django models..., just does not use views and templates).
Database backend is SQLite3.

This applications also needs to run some jobs periodically. I wrote a unix-like daemon that uses python-gobject and python-glib, and runs those jobs like this:

gobject.timeout_add_seconds(seconds, someCallback...)
gobject.timeout_add_seconds(seconds, someCallback...)
...
gobject.timeout_add_seconds(seconds, someCallback...)

glib.MainLoop().run()

I tested it, and there are some strange problems on written data in sqlite db. I think that's because there are two Python instances reading and writing from/to a single sqlite db. One for apache+wsgi and one for my own daemon. (Or event 3 Python instances, when I use command line interface)

My question is, what do recommend me to do? Put those timeout_add and MainLoop in my "dj_survey.wsgi" to run on apache start?

Wouldn't Celery be an option for you? The [periodic tasks](http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html) functionality seems quite suited to your needs. — Mathieu Dhondt, Feb 15 '12 at 14:55
Yeah, I'm reading about it. But this topic doubted me a bit http://stackoverflow.com/questions/8404325/django-celery-works-in-development-fails-in-wsgi-production-how-to-debug — saeedgnu, Feb 16 '12 at 07:24
Since I don't need too many features! Just something minimalist and stable. — saeedgnu, Feb 16 '12 at 07:32
Let me change the question: Can I run a jsonrpc server within wsgi environment? — saeedgnu, Feb 16 '12 at 07:43
I saw wsgi-jsonrpc, But I'm afraid django doesn't allow me to have two wsgi applications! — saeedgnu, Feb 16 '12 at 08:06

score 3 · Accepted Answer · answered Feb 15 '12 at 13:13

3

No, you don't want to run background processes inside your apache/whatever WSGI environment.

Start them on the shell and use some method to communicate with your background process.

answered Feb 15 '12 at 13:13

ThiefMaster

310,957
84
592
636

Thanks, What method should I use to communicate? (I suppose not using D-BUS for server!) – saeedgnu Feb 15 '12 at 13:16
Should I send http requests from that daemon to apache/django from localhost? (Because apache and then django only responds to these requests) – saeedgnu Feb 15 '12 at 13:22
Or maybe I better not use WSGI at all. – saeedgnu Feb 15 '12 at 13:27

Python, Django and event loop (periodic jobs)

1 Answers1