42

I guess this should be pretty elementary but I've tried to google it and I've read the docker documentation. However, I still can't grasp what exactly does "Thin Pool" mean and the role it plays in the docker world.

Anthony Geoghegan
  • 11,533
  • 5
  • 49
  • 56
Antonio Gomez Alvarado
  • 1,842
  • 2
  • 13
  • 24
  • 3
    It's rare that I upvote such short questions but in this case the official documentation is not obvious and I couldn't easily find any other clear explanation so I think this is a great question for [so]. – Anthony Geoghegan Mar 28 '18 at 10:15

1 Answers1

58

Short story:

A thin pool is a storage source that provides on-demand allocation for storage space. It is more or less similar to virtual memory, which provides full address space to every process.

Long story:

Fat Provisioning

The traditional storage allocation method is called "fat" or "thick" provisioning.

For example, a user claims to use 10G storage space. Fat provisioning then reserves 10G physical storage space for this user even though he/she only uses 1% of it. No one else can use this reserved space.

Thin Provisioning

Thin provisioning provides a mechanism of on-demand storage allocation, which allows a user to claim more storage space than has been physically reserved for that user.

In other words, it enables over-allocation for storage space. Think about RAM's over-commit feature.

Thin Pool

Thin pool is a conceptional term which stands for the backing storage source used by thin provisioning. Thin provisioning allocates virtual chunks of storage from thin pool, while fat provisioning allocates physical blocks of storage from the traditional storage pool.

Thin Pool in Docker

The Docker Engine can be configured to use Device Mapper as its storage driver. This is where you deal with thin provisioning. According to Docker's documentation:

Production hosts using the devicemapper storage driver must use direct-lvm mode. This mode uses block devices to create the thin pool.

Two different spaces of thin pool need to be taken care of: the Metadata space (which stores pointers) and the Data space (which stores the real data). At the very beginning, all the pointers in Metadata space point to no real chunks in the pool. No chunk in data space is really allocated until a write request arrives. This is nothing new if you are familiar with the virtual memory mechanism.

Let's take a look at the output of docker info:

Data Space Used: 11.8 MB
Data Space Total: 107.4 GB
Data Space Available: 7.44 GB
Metadata Space Used: 581.6 kB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.147 GB
Thin Pool Minimum Free Space: 10.74 GB

Here, the only confusing one is the Thin Pool Minimum Free Space. What does it stand for?

It specifies the min free space in GB in a thin pool required for a new device creation to succeed. This check applies to both free data space as well as free metadata space.

Container creation (during docker pull or docker run) fails if free space in thin pool is less than the value in Thin Pool Minimum Free Space. Insufficient space requires either adding more storage into thin pool or clearing up unused images.

Links:

slm
  • 15,396
  • 12
  • 109
  • 124
Yuankun
  • 6,875
  • 3
  • 32
  • 34
  • Thanks @Yuakun for shedding more light on this but still how does this play a role for docker ? Whenever docker says that " Thin Pool Minimum Free Space " is XGB does it mean that docker needs thin pool to have at least XGB left from the pool to operate ? Or is this actually increasing the pool size to a minimum XGB ? ( Which means that more chunks are available for docker to use ) – Antonio Gomez Alvarado Mar 28 '18 at 13:38
  • Added a "Thin Pool in Docker" section. – Yuankun Mar 28 '18 at 15:09
  • So the "Thin Pool Minimum Free Space: 10.74 GB" is actually 10% of "Data Space Total: 107.4 GB" ? Is it correct to say that the 'whole pool' is actually the total Data Space? – Antonio Gomez Alvarado Mar 29 '18 at 08:47
  • 3
    No, whole pool = Data Space + Metadata Space. In this case, `Thin Pool Minimum Free Space` stands only for Data Space. Actually there is a `Thin Pool Minimum Free Metadata Space` which is 10% of "Metadata Space Total: 2.147 GB". Both limitations will be checked when creating containers. But where does the `10%` come from? It is a tunable option for the Docker Engine, `dm.min_free_space`, the unit is percentile. – Yuankun Mar 29 '18 at 09:40
  • Thanks Yuankun - this makes a whole lot more sense now! – Antonio Gomez Alvarado Mar 29 '18 at 09:54
  • Thanks for this answer! This answer can be improved by adding some history on the word "pool" as it is a non-natural word to use for this. – Elijah Lynn Feb 18 '21 at 01:06