Gunicorn config.py file with 2 workers and 2 threads handles just 2 requests in parallel

Question

There is no answer in the original github issue: https://github.com/tiangolo/meinheld-gunicorn-flask-docker/issues/48

In the simple setup below I am expecting the gunicorn to process at least 4 requests in parallel, but I get just 2. In the configuration below we can see 2 workers with 2 threads.

In addition, we tried to configure number of workers == 1 with 2 threads, then we got just 1 request being processed in a given moment. So it's looks like gunicorn + meinheld setup is able to process just 1 request in each worker, which is way inefficient.

We are sending 53 requests in parallel in each setup.

simple_app.py:

import sys

from flask import Flask, request
import json
import time

app = Flask(__name__)


@app.route("/", methods=['POST'])
def hello():
    headers = request.headers
    data = json.loads(request.data)
    version = "{}.{}".format(sys.version_info.major, sys.version_info.minor)
    message = "Hello World from Flask in a Docker container running Python {} with Meinheld and Gunicorn (default): {}:{}".format(
        version, headers, json.dumps(data)
    )
    print(message)
    time.sleep(15)
    return message

gunicorn_dev_conf.py:

import json
import multiprocessing
import os
workers_per_core_str = os.getenv("WORKERS_PER_CORE", "2")
web_concurrency_str = os.getenv("WEB_CONCURRENCY", None)
host = os.getenv("HOST", "0.0.0.0")
port = os.getenv("PORT", "80")
bind_env = os.getenv("BIND", None)
use_loglevel = os.getenv("LOG_LEVEL", "debug")
if bind_env:
  use_bind = bind_env
else:
  use_bind = "{host}:{port}".format(host=host, port=port)
cores = multiprocessing.cpu_count()
workers_per_core = float(workers_per_core_str)
default_web_concurrency = workers_per_core * cores
if web_concurrency_str:
  web_concurrency = int(web_concurrency_str)
  assert web_concurrency > 0
else:
  web_concurrency = int(default_web_concurrency)
# Gunicorn config variables
loglevel = use_loglevel
workers = web_concurrency
workers = 2
reuse_port = True
bind = use_bind
keepalive = 120
errorlog = "-"
threads = 2
# For debugging and testing
log_data = { "loglevel": loglevel, "workers": workers, "bind": bind,
# Additional, non-gunicorn variables
   "workers_per_core": workers_per_core, "host": host, "port": port}
accesslog = "-"
preload_app = False
print_config =  True
# max_requests = 1900
# max_requests_jitter = 50
sendfile = True
print(json.dumps(log_data), flush=True)

docker run command:

docker run --rm --name demo_dev --privileged -it -p 8080:80 -v ./gunicorn_dev_conf.py:/app/gunicorn_conf.py:rw -v ./simple_app.py:/app/main.py tiangolo/meinheld-gunicorn-flask:python3.8-alpine3.11

Parallel curl command:

for i in {1..53}
do
echo "Welcome num$i; $(date)"
curl "http://localhost:8080/" -d @simpleJson.json -H 'content-type: application/json' -o dir_$i&&echo "num"$i"; $(date)" &
done

simpleJson.json:

{
 "field1": [
     {
         "field2": "09-090"
     }
 ]
}

output:

10.0.2.100 - - [05/Aug/2021:12:16:20 +0000] "POST / HTTP/1.1" 200 251 "-" "curl/7.61.1"
Hello World from Flask in a Docker container running Python 3.8 with Meinheld and Gunicorn (default): Host: localhost:8080
User-Agent: curl/7.61.1
Accept: */*
Content-Type: application/json
Content-Length: 70

:{"field1": [{"field2": "09-090"}]}
10.0.2.100 - - [05/Aug/2021:12:16:20 +0000] "POST / HTTP/1.1" 200 251 "-" "curl/7.61.1"
Hello World from Flask in a Docker container running Python 3.8 with Meinheld and Gunicorn (default): Host: localhost:8080
User-Agent: curl/7.61.1
Accept: */*
Content-Type: application/json
Content-Length: 70

:{"field1": [{"field2": "09-090"}]}

>> WAIT 15 seconds...


10.0.2.100 - - [05/Aug/2021:12:16:35 +0000] "POST / HTTP/1.1" 200 251 "-" "curl/7.61.1"
Hello World from Flask in a Docker container running Python 3.8 with Meinheld and Gunicorn (default): Host: localhost:8080
User-Agent: curl/7.61.1
Accept: */*
Content-Type: application/json
Content-Length: 70

:{"field1": [{"field2": "09-090"}]}
10.0.2.100 - - [05/Aug/2021:12:16:35 +0000] "POST / HTTP/1.1" 200 251 "-" "curl/7.61.1"
Hello World from Flask in a Docker container running Python 3.8 with Meinheld and Gunicorn (default): Host: localhost:8080
User-Agent: curl/7.61.1
Accept: */*
Content-Type: application/json
Content-Length: 70

:{"field1": [{"field2": "09-090"}]}