Hide node core count from docker container

Question

I am currently building a system that run services in individual deployments in a kubernetes cluster and require the limitation of visible cores inside the container.

Node Specs:
8 Cores
16 GB memory

When I deploy, log into the container and inspect the core count with python's multiprocessing module I get the following output:

In [1]: import multiprocessing
In [2]: multiprocessing.cpu_count()
Out[2]: 8

This output is correct seeing that my node has 8 cores.

What I would like is for the container to return a core count of 1 as the core count when running the same multiprocessing.cpu_count() command.

The example use case for this is as follows:

Developer creates an api services running python and gunicorn, gunicorn's default behaviour is to create workers for the number of cores available.

The above will be a problem when the pod is only allowed 100m CPU and 300m Mem as the 8 workers will at some point fill up the memory and the pod will crash, I also would like to hide this for the sake of potential confusion and unexpected behaviour when detecting resources programmatically.

I have tried using the --cpuset-cpus=0 flag as it stood out as a potential solution, but from what I understand and experienced is that this will merely limit the container to the specified core, which does not hide the other 7 cores.

Is there any way of doing this, any help will be appreciated.

Try this article it my help you how to set docer cpu limit https://stackoverflow.com/questions/26841846/how-to-allocate-50-cpu-resource-to-docker-container — Ghassen, Jun 21 '19 at 10:45
--cpuset is now --cpuset-cpus from what I can see and --cpus don't work either :/ — Illegal Operator, Jun 21 '19 at 14:19
if you always up for a single worker, why not just to set number of gunicorn workers in config i.e. `gunicorn --workers=1 --threads=1 main:app` — A_Suh, Jun 26 '19 at 10:55
Yah you can do that, and that is fine, but as i pointed out if an app uses an automated/dynamic method for scaling up threads it will see 8 cores and create 8 threads while only having a fraction of 1 core — Illegal Operator, Jun 26 '19 at 13:41

score 0 · Answer 1 · answered Mar 29 '21 at 13:27

You could have an environment variable NUM_CPUS that if it's present you should use it instead of multiprocessing.cpu_count(). Something like:

import os

num_cpus = os.getenv("NUM_CPUS")
if num_cpus is not None:
    cpu_count = max(round(num_cpus), 1)
else:
    cpu_count = multiprocessing.cpu_count()

Hide node core count from docker container

1 Answers1