3

In using ./manage.py shell to exercise some code, I've come across something I don't understand.

Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from multiprocessing import Process
>>> from django.core.management import call_command
>>> p = Process(target=call_command("processphoto", 1000))
Successfully populated photo "1000"  (This is output from my processphoto command)
>>>

I never get the opportunity to do a p.start() or set any other variables on the process. It appears to execute on instantation. When I try to use the code in my view, I don't appear to get multiple processes spawning at all, everything remains on the one core.

What am I doing wrong, or misunderstanding? I want to generate separate manage.py processphoto commands in separate processes to fully utilize the multi-core server.

William Maness
  • 91
  • 1
  • 7

1 Answers1

2

call_command(...) calls the function. Instead, pass the function object, call_command, and its arguments separately to Process:

 p = Process(target=call_command, args = ("processphoto", 1000))
unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677
  • Doh! Good lord, you'd think I was some first time newbie... Thanks. – William Maness Nov 05 '12 at 19:46
  • No worries. You see this once and know it forever. – unutbu Nov 05 '12 at 19:48
  • Please note, doing this will copy the database connection between all children, and can cause problems. I only bring this up since all of my command scripts have some sort of database interaction. Here is a thread which sheds some light on the problem, and makes some suggestions to help: [Django multiprocess database connection][1] [1]: http://stackoverflow.com/questions/8242837/django-multiprocessing-and-database-connections – Luke Dupin May 21 '15 at 19:07
  • The problem you warn about occurs whenever a process is spawned from a module with an open connection defined in the module's globally namespace. This question does not mention any open database connections. So while your warning is True, I don't think it has much relevance to the question or answer dealt with here. – unutbu May 21 '15 at 21:09