Docker build command creates large amount of folders in windowsfilter folder: what's the purpose and how to control

Question

I'm building a new image like so docker build . -t test and copy contents from host OS folder into it:

Dockerfile

FROM mcr.microsoft.com/windows/servercore:ltsc2019
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

RUN mkdir root
RUN cd root
WORKDIR /root
RUN mkdir test
COPY test /root/test
    #USING 'COPY' here instead of 'ADD' because of https://stackoverflow.com/questions/24958140/what-is-the-difference-between-the-copy-and-add-commands-in-a-dockerfile

I get this output in Powershell:

PS D:\Programs> docker build . -t test
Sending build context to Docker daemon  1.644GB
Step 1/7 : FROM mcr.microsoft.com/windows/servercore:ltsc2019
ltsc2019: Pulling from windows/servercore
4612f6d0b889: Pull complete
c67ded6868b6: Pull complete
Digest: sha256:1be9c8378b8c32d31538253a4b96169f5138a5f6d4c0a04e8d8f9a80b9ac8097
Status: Downloaded newer image for mcr.microsoft.com/windows/servercore:ltsc2019
 ---> d1724c2d9a84
Step 2/7 : SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
 ---> Running in 1bfefefbe433
Removing intermediate container 1bfefefbe433
 ---> 37de702deb33
Step 3/7 : RUN mkdir root
 ---> Running in e26d6b49ced7


    Directory: C:\


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----         9/8/2021   1:07 PM                root


Removing intermediate container e26d6b49ced7
 ---> 451c4d3f3ea1
Step 4/7 : RUN cd root
 ---> Running in 74a228f8118f
Removing intermediate container 74a228f8118f
 ---> 3f175ac67f1d
Step 5/7 : WORKDIR /root
 ---> Running in 5f783d5b2332
Removing intermediate container 5f783d5b2332
 ---> 68b24e033f87
Step 6/7 : RUN mkdir test
 ---> Running in 5771bb7a593a


    Directory: C:\root


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----         9/8/2021   1:08 PM                test


Removing intermediate container 5771bb7a593a
 ---> 35fa0b2af157
Step 7/7 : COPY test /root/test
 ---> 60eab8242865
Successfully built 60eab8242865
Successfully tagged test:latest

It creates EIGHT(!) folders on the host OS in folder "\docker\windowsfilter"

This seems excessive, it looks like a folder for each step in my Dockerfile, so what are all these folders for and could I prevent creation/autodelete the unneeded ones to keep everything tidy?

Host OS: Windows Server 2019 standard Version 1809 17763.2114
Docker: version 20.10.4, build 110e091

Please note that I do NOT use the Docker Desktop application, see here how I installed Docker.

Update 1

I updated my Dockerfile based on @Noam's suggestion like so:

#RUN mkdir root
#RUN cd root
#WORKDIR /root
WORKDIR /root #creates root directory if not exists, then enters it

After this change, 6 folders (instead of 8) are created in windowsfilter, so my hunch that Docker creates a folder for each command in Dockerfile seems to be correct.

Answer 1

Hallo @Flo this issue seems to be a long time issue specifically with docker desktop,

Causes:

One option that some times seems to explain it is upgrading from older versions of docker desktop and the software not cleaning up old directories.

Another I that docker saves built image layers there so if you have many versions of the same image your dangling ones could reside there requesting a run of docker image prune or maybe even docker system prune

Solutions:

An opensource cleanup script has been published and it seems to help a great deal, its called docker-ci-zap and for the most part it appears to handle it quite well.

if the issue still persists there is a fix suggested in the docker for desktop github issue page is adding -removing suffix to the folder names in the windowsfilter directory followed by a restart to allow docker to cleanup the unneeded folders

if it's still causing issues try returning to factory deafults and if that doesn't do it either maybe a fresh docker for desktop install might seal the deal though even then some experienced some difficulties so eventually if it really bothers and you need to use it, reinstalling windows usually looks like the final blow.

suggestion for Dockerfile improvments

in your Dockerfile the lines describing the folders might cause problems up a head du to unabsolute paths and improper use of the Dockerfile command.

replace:

RUN mkdir root
RUN cd root
WORKDIR /root

with:

WORKDIR /root

it achives the same purpose (creating and entering the new (if didn't exist yet) directory)

what does docker-ci-zap do?

You can get a rough sense by looking at the code used to compile the exe file that resides in the GitHub repo:

it uses the hcsshim driver and its DestroyLayer function that deletes the layer files from the host disk, or as the docs put it:

DestroyLayer will remove the on-disk files representing the layer with the given path, including that layer's containing folder, if any.