Docker squash and layer: which layers are merged and which are still shared

Question

Hint: I am aware of this question but it does not exactly / fully answer my question or more or less not targeting it (but you could derive some of it - i would say there should be a yes/no question for that - here it is ).

CASE:

Assuming i have 3 Images, one called BASE, one CHILDa and CHILDb, both childs do FROM BASE.
Assume BASE has a size of 1GB and assuming we do not know or are particular interested if it is squashed or not, since it should not matter ( IMHO )
CHILDa and CHILDb both add 10 layers with a size of each 500MB
Assume we used docker build --squash CHILDa when creating CHILDa and CHILDb

Question:

When pulling CHILDa and CHILDb from the registry, i understand that BASE layers will be pulled first. Now, my question is, what is the exact size of the images on the drive:

a) 1GB(base) + 500MB(CHILDa) + 500MB(childB)=2GB
b) (1GB + 500MB) + (1GB + 500MB)=3GB

So are the layers of BASE shared as in non --sqaush cases ( this would be a) then ) or are they not shared, b) then

I understand, that layers from BASE should not be squashed when CHILDa is build and squashed, only the layers created in CHILDa are squashed to one layer, so the history should be looking like this

BASE LAYER1
BASE LAYER2
BASE LAYER3
...
CHILDa LAYER1 (squashed)

So that means, all the BASE layers should be shared with CHILDb on transfer and also shared in terms of disk space when CHILDa and CHILDb are both pulled. This would mean a) would be the answer.

I am asking this question to have a definite answer, not looking for a suggestion or an implication based on the docs. Probably even backed up having a test? It would be not the first time that docs and technical implementation are not matching eachother( in docker)

score 6 · Accepted Answer · answered Oct 30 '17 at 14:12

The answer is A. You can see what Docker is doing easily enough with an inspect on the resulting image:

First build two images:

$ docker build -t jenkins-blueocean:full .
$ docker build -t jenkins-blueocean:squash --squash .

Compare the total disk space used on the for an image (which counts the base image, in this case jenkins/jenkins):

$ docker image ls jenkins/jenkins:2.77         
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
jenkins/jenkins     2.77                1a057287c665        6 weeks ago         814MB
$ docker image ls jenkins-blueocean:full
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
jenkins-blueocean   full                773f9e1cbd94        3 minutes ago       1.29GB
$ docker image ls jenkins-blueocean:squash
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
jenkins-blueocean   squash              9a8816dcc900        2 minutes ago       1.28GB

This disk space is cumulative, and will double count layers used in different images. So we need to look at the actual layers. Compare the layers for the three images (base, full, and squashed) using a docker inspect:

$ docker inspect -f '{{json .RootFS.Layers}}' jenkins/jenkins:2.77 | jq .
[
  "sha256:45f0f161f0749d09482ed1507925151b22b1f8c0c85970fe0857d61e530264b4",
  "sha256:560ec518567f4117ed651db78b9c46eee39e00f38a87d6200ad7c87b79432064",
  "sha256:deccd4ec00609f5f711578af469ce4ff43a5c73efc52517fc8ca362ebd36860c",
  "sha256:23543e96fe44ca57a96d8552a6d9d218f7aa93b928a1ec8bafcaa9df3cc5723b",
  "sha256:3de9ccb39b3bcf90c9215a49a84b340fedad87840d0580ffe0f0e0e8a1cb1f53",
  "sha256:559298d0ee994bb9f12a77b1acc6fdfb6c7120cbcadfd640f7a9d171729b2cb1",
  "sha256:4dc9d0cb0b3ca0f565aa29c7762f7322ece1e1fb51711feac3a52f3c20a28d2f",
  "sha256:93d818bcd1d5eb6c689e6964e89feb8a8a3a394a998552c540b7db74986266c7",
  "sha256:ac3d4345fe0474e18265fbb999fe6ab1c077fbb59876317406c7974c75c7ab5d",
  "sha256:83a60a36cc44ca6fdab64823e805a853106be334239eb9d43cc1b220bb6ad238",
  "sha256:7c78d70f156aaaee25540c9100ca28b68b554a966d448079896c413ae71a0e5d",
  "sha256:cfd8defeb8a79686260691ce89a36772b21af0f736628492c835bb8a5740b817",
  "sha256:fc4dc905efd22f932b74f95b53904736bebf52c2033e9853c54efb0b3f01560f",
  "sha256:456fa2e1bb798ba4ccc5d433013973772b673dfff1f1386f18ceffe7d18132da",
  "sha256:2446924bd5315bf6c46e8a5db2b61247da4ded48f4de148c15f8f5a2f9b1e91a",
  "sha256:5a4416e8de72a14e97b53484e6016cc8a5b79398a25eb3b80fa099740b9f32e3",
  "sha256:20901b1036e739e01c99d83e461059b3974003835a31e473f348fd70ece6c4e3",
  "sha256:6d9d9244ead270d545d5de253a6ebb95398a7c63b10977c5cf7345b1cbf7d201",
  "sha256:056fab22f880b32a4bbe4725b5b227891290097fd3791af452e52eb98b02cfb4",
  "sha256:18a8691ee145f81f617bf788f39617f46d84b9924911317e6226139074b1f3e1"
]

This base image goes up to layer 18a8691... Compare that with the full:

$ docker inspect -f '{{json .RootFS.Layers}}' jenkins-blueocean:full | jq .
[
  "sha256:45f0f161f0749d09482ed1507925151b22b1f8c0c85970fe0857d61e530264b4",
  "sha256:560ec518567f4117ed651db78b9c46eee39e00f38a87d6200ad7c87b79432064",
  "sha256:deccd4ec00609f5f711578af469ce4ff43a5c73efc52517fc8ca362ebd36860c",
  "sha256:23543e96fe44ca57a96d8552a6d9d218f7aa93b928a1ec8bafcaa9df3cc5723b",
  "sha256:3de9ccb39b3bcf90c9215a49a84b340fedad87840d0580ffe0f0e0e8a1cb1f53",
  "sha256:559298d0ee994bb9f12a77b1acc6fdfb6c7120cbcadfd640f7a9d171729b2cb1",
  "sha256:4dc9d0cb0b3ca0f565aa29c7762f7322ece1e1fb51711feac3a52f3c20a28d2f",
  "sha256:93d818bcd1d5eb6c689e6964e89feb8a8a3a394a998552c540b7db74986266c7",
  "sha256:ac3d4345fe0474e18265fbb999fe6ab1c077fbb59876317406c7974c75c7ab5d",
  "sha256:83a60a36cc44ca6fdab64823e805a853106be334239eb9d43cc1b220bb6ad238",
  "sha256:7c78d70f156aaaee25540c9100ca28b68b554a966d448079896c413ae71a0e5d",
  "sha256:cfd8defeb8a79686260691ce89a36772b21af0f736628492c835bb8a5740b817",
  "sha256:fc4dc905efd22f932b74f95b53904736bebf52c2033e9853c54efb0b3f01560f",
  "sha256:456fa2e1bb798ba4ccc5d433013973772b673dfff1f1386f18ceffe7d18132da",
  "sha256:2446924bd5315bf6c46e8a5db2b61247da4ded48f4de148c15f8f5a2f9b1e91a",
  "sha256:5a4416e8de72a14e97b53484e6016cc8a5b79398a25eb3b80fa099740b9f32e3",
  "sha256:20901b1036e739e01c99d83e461059b3974003835a31e473f348fd70ece6c4e3",
  "sha256:6d9d9244ead270d545d5de253a6ebb95398a7c63b10977c5cf7345b1cbf7d201",
  "sha256:056fab22f880b32a4bbe4725b5b227891290097fd3791af452e52eb98b02cfb4",
  "sha256:18a8691ee145f81f617bf788f39617f46d84b9924911317e6226139074b1f3e1",
  "sha256:679b85b8d42598a7ecb5988e408da49cbb3f86402fd2e5694104839ff17a7015",
  "sha256:5fa620489d92edd3e7922d9335d803ea83c148793044e0da99144152f7988437",
  "sha256:17d03c6eda4a4d989f6751bb53d7bf356309938a1076af75bdf440195471fa2b",
  "sha256:7a78b2c7c995ddab1ba675aba1c2bc54cc289ba148fd39b600f592060d98c459",
  "sha256:f56b6c3fd8713236d077a95568a58445e6d6423113c0b68c6f10bef39bd6b6ff"
]

The full image added 5 layers to the image. Viewing the squashed:

$ docker inspect -f '{{json .RootFS.Layers}}' jenkins-blueocean:squash | jq .
[
  "sha256:45f0f161f0749d09482ed1507925151b22b1f8c0c85970fe0857d61e530264b4",
  "sha256:560ec518567f4117ed651db78b9c46eee39e00f38a87d6200ad7c87b79432064",
  "sha256:deccd4ec00609f5f711578af469ce4ff43a5c73efc52517fc8ca362ebd36860c",
  "sha256:23543e96fe44ca57a96d8552a6d9d218f7aa93b928a1ec8bafcaa9df3cc5723b",
  "sha256:3de9ccb39b3bcf90c9215a49a84b340fedad87840d0580ffe0f0e0e8a1cb1f53",
  "sha256:559298d0ee994bb9f12a77b1acc6fdfb6c7120cbcadfd640f7a9d171729b2cb1",
  "sha256:4dc9d0cb0b3ca0f565aa29c7762f7322ece1e1fb51711feac3a52f3c20a28d2f",
  "sha256:93d818bcd1d5eb6c689e6964e89feb8a8a3a394a998552c540b7db74986266c7",
  "sha256:ac3d4345fe0474e18265fbb999fe6ab1c077fbb59876317406c7974c75c7ab5d",
  "sha256:83a60a36cc44ca6fdab64823e805a853106be334239eb9d43cc1b220bb6ad238",
  "sha256:7c78d70f156aaaee25540c9100ca28b68b554a966d448079896c413ae71a0e5d",
  "sha256:cfd8defeb8a79686260691ce89a36772b21af0f736628492c835bb8a5740b817",
  "sha256:fc4dc905efd22f932b74f95b53904736bebf52c2033e9853c54efb0b3f01560f",
  "sha256:456fa2e1bb798ba4ccc5d433013973772b673dfff1f1386f18ceffe7d18132da",
  "sha256:2446924bd5315bf6c46e8a5db2b61247da4ded48f4de148c15f8f5a2f9b1e91a",
  "sha256:5a4416e8de72a14e97b53484e6016cc8a5b79398a25eb3b80fa099740b9f32e3",
  "sha256:20901b1036e739e01c99d83e461059b3974003835a31e473f348fd70ece6c4e3",
  "sha256:6d9d9244ead270d545d5de253a6ebb95398a7c63b10977c5cf7345b1cbf7d201",
  "sha256:056fab22f880b32a4bbe4725b5b227891290097fd3791af452e52eb98b02cfb4",
  "sha256:18a8691ee145f81f617bf788f39617f46d84b9924911317e6226139074b1f3e1",
  "sha256:e05668bb7cbab8f964ea3512a9ce41568330218e0e383693ad9edfd1befce9aa"
]

It only added a single new layer. The base image itself was not squashed.

On disk, each layer is only stored once, so you only count the base image once for disk usage.

Note, I do not recommend squashing images in most scenarios. It breaks the value of image layer caching of earlier layers inside the image. Instead, I recommend organizing the Dockerfile to maximize the value of layer caching, and using multi-stage builds to get a layer down to a single copy if there's some systemic overhead.

excellent answer ans also a good guide how to approve it yourself. Thank you BMitch! — Eugen Mayer, Oct 30 '17 at 15:25
just as a side note, in nearly all my tests, even on the images with about 80 layers on top, squashing had a very little impact, even if the image size was to be considered rather large ( 1.5GB ). I think it still comes down to using the best practise to cleanup your operation right after you did something as the only way to keep the image small. Like `apt-get update && install && rm /var/lib/apt && autoremove` all in one line, including installing build-essentials and removing them or better - create a builder-image to build the artifact and just install on the image — Eugen Mayer, Oct 30 '17 at 16:16
The real value of squashing is if you include lots of data in one layer that you change or delete later. If you follow best practices by merging that into one RUN step, the value goes away and you're left with the downside of no layer caching. — BMitch, Oct 30 '17 at 16:19

Docker squash and layer: which layers are merged and which are still shared

1 Answers1

Linked