What is the appropriate number of Gunicorn workers for each Amazon Instance Type?

Question

I'm currently trying to figure out what the appropriate number of workers is for each Amazon Instance Type. I used to run one Gunicorn worker, however that proved to be quite slow.

Many developers are currently using this formula to gauge how many workers would be suitable:

NUM_WORKERS=3  #recommended formula here is 1 + 2 * NUM_CORES

The problem I'm having is that Amazon isn't quite clear as to the number of cores each instance is running. For example, an M1 Small Instance has 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit)

What does that essentially mean? That it has one core? or that it has two cores?

dhilipsiva · Accepted Answer · 2015-10-22T15:46:44.253

43

I know this is a old question. But I think I have a better answer to this question. Gunicorn docs suggests that 2n+1 [gunicorn -w <2n+1> myapp:wsgi] is a good guess for number of workers (Yes, n = number of cores). I came up with a tiny shell script to apply this formula. All you need to do is this:

gunicorn -w $(( 2 * `cat /proc/cpuinfo | grep 'core id' | wc -l` + 1 )) myapp:wsgi

Where the command

cat /proc/cpuinfo | grep 'core id' | wc -l

will return the total number of actual CPU cores (n). So

$(( 2 * `cat /proc/cpuinfo | grep 'core id' | wc -l` + 1 ))

equates to 2n+1 formula.

This will apply 2n+1 formula to all the linux-based machines. You dont need to know the number of workers for each type of instance or anything like that.

Reference: http://dhilipsiva.com/2015/10/22/appropriate-number-of-gunicorn-workers.html

edited Oct 22 '15 at 15:46

answered Dec 27 '14 at 03:20

dhilipsiva

3,688
1
24
35

11

**gunicorn.conf.py** import multiprocessing; workers = multiprocessing.cpu_count() * 2 + 1 http://docs.gunicorn.org/en/19.3/configure.html – kev Oct 18 '15 at 22:54
@kev so is using a .py instead of a gunicorn.service file advised? – Mr-Programs Jan 27 '19 at 00:09
@Mr-Programs not sure anymore what filetype it is, but it is (still) in the official documentation: https://docs.gunicorn.org/en/19.9.0/configure.html – kev Jan 27 '19 at 09:16
4

Why not simply use `$(( 2 * \`nproc\` + 1 ))` to get the number of processing units available? – Faheel Jul 29 '19 at 09:53
@Faheel, did not know that such a command existed :P – dhilipsiva May 25 '21 at 14:18
aws ec2 advertises threads, not the physical cores. ex, a 4-core cpu that supports hyperthreading has 8 cores, which is what aws advertises, you can check it by doing `cat /proc/cpuinfo | grep 'cores'`. with this answer, i am now confused, should i use "(2 * n-cores) + 1" or "(2 * n-threads) + 1"? – Naveen Reddy Marthala Jan 15 '22 at 09:29
@NaveenReddyMarthala More accurately, each thread is a vCPU core in EC2. i.e number of vCPU cores = number of threads. So they are one and the same as far as EC2 is concerned. – dhilipsiva Jan 16 '22 at 15:31
yes, but do I use one workder per cpu-thread or vCPU or one worker per core(2 threads)? – Naveen Reddy Marthala Jan 16 '22 at 15:33
@NaveenReddyMarthala My guess is, 2*vCPU + 1; Not entirely sure if its the right thing to do. – dhilipsiva Feb 10 '22 at 09:09

score 22 · Answer 2 · edited May 23 '17 at 12:17

22

Building on epicbrew's work, here's how to launch 2N+1 Gunicorn workers, where N = number of CPU cores:

gunicorn --workers=$((2 * $(getconf _NPROCESSORS_ONLN) + 1)) wsgi:application

This works on both Linux and macOS! More detail on the John Tells All blog.

edited May 23 '17 at 12:17

Community

1
1

answered Jul 29 '16 at 20:22

johntellsall

14,394
4
46
40

score 8 · Answer 3 · answered Apr 12 '13 at 19:53

The Amazon EC2 m1.small instance type definitely has one virtual core only; from a threading/worker perspective you can ignore the EC2 Compute Unit (ECU) specification entirely and take the listed number of (virtual) cores on the Amazon EC2 Instance Types page literally, multiplying by the number of listed CPUs, where applicable (only relevant for cluster instances).

If you want to avoid doing the math and/or have programmatic access to that information, you might want to have a look at the missingcloud project - the aws.json dataset features a cores field within the instance_types collection, e.g.:

"instance_types" : {
        "m1.small" : {
            "compute_units" : 1,
            "cores" : 1,
            "gpus" : 0,
            "ramMB" : 1700,
            "storageGB" : [10, 160],
            "i/o" : "moderate",
            "ebs_optimized_iopsMbps" : 0,
            "arch" : [32,64]},
        ...
}

why do you advise to not get this info programatically? gunicorn documentation even has it done programmatically — Mr-Programs, Jan 27 '19 at 00:18

score 3 · Answer 4 · answered Apr 13 '13 at 07:06

3

An easy way to see how many cpu's are detected is to run top and press "1" to show the number of cpu's. You will see cpu0, cpu1, cpu2 etc.

answered Apr 13 '13 at 07:06

Lars Hansson

366
1
3

What is the appropriate number of Gunicorn workers for each Amazon Instance Type?

4 Answers4