celery always has 3 processes, why?

Question

I run the crawler application with celery(version 3.1) tasks and django(version 1.6.5. using its orm). And the command is:

celery -A crawlerproj worker -P gevent -c 1000 -l info

by using the 'htop' in Ubuntu, I see that celery always has 3 processes. And there are 5 more mysqldb processes. I don't know why. Can numbers of celery and mysqldb decrease?

My vps has 1G Memory and 1 core. Here is the snapshot of the htop result.

http://stackoverflow.com/questions/12213665/celery-decrease-number-of-processes ? — coldmind, Sep 02 '14 at 11:34

score 0 · Answer 1 · answered Sep 02 '14 at 17:05

0

By default, celery starts as many worker processes as it thinks you have processors/cores/threads: http://celery.readthedocs.org/en/latest/configuration.html

If you’re doing mostly I/O you can have more processes, but if mostly CPU-bound, try to keep it close to the number of CPUs on your machine. If not set, the number of CPUs/cores on the host will be used.

answered Sep 02 '14 at 17:05

Dan Lenski

76,929
13
76
124

but my vps just have 1 core. so you mean my vps have 3CPU? – no13bus Sep 04 '14 at 15:13
I don't know how Celery guesses the number of CPUs/cores/threads for a VPS. In any case, you can easily change the default as shown in the docs. – Dan Lenski Sep 04 '14 at 15:17
And I have another question to ask. http://stackoverflow.com/questions/25955798/can-celerys-beat-tasks-execute-at-timed-intervals – no13bus Sep 21 '14 at 08:23

celery always has 3 processes, why?

1 Answers1