Copy contents from host OS into Docker image without rebuilding image

Question

I'm building a new image and copy contents from host OS folder D:\Programs\scrapy into it like so: docker build . -t scrapy

Dockerfile

FROM mcr.microsoft.com/windows/servercore:ltsc2019
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

RUN mkdir root
RUN cd root
WORKDIR /root
RUN mkdir scrapy
COPY scrapy to /root/scrapy

    

Now when I add new contents to the host OS folder "D:\Programs\scrapy" I want to also add it to image folder "root/scrapy", but I DON'T want to build a completely new image (it takes quite a while). So how can I keep the existing image and just overwrite the contents of the image folder "root/scrapy".
Also: I don't want to copy the new contents EACH time I run the container (so NOT at run-time), I just want to have a SEPARATE command to add more files to an existing image and then run a new container based on that image at another time.

I checked here: How to update source code without rebuilding image (but not sure if OP tries to do the same as me)

UPDATE 1

Checking What is the purpose of VOLUME in Dockerfile and docker --volume format for Windows

I tried the commands below, all resulting in error:

docker: Error response from daemon: invalid volume specification: ''. See 'docker run --help'.

Where <pathiused> is for example D:/Programs/scrapy:/root/scrapy

docker run -v //D/Programs/scrapy:/root/scrapy scrapy

docker run -v scrapy:/root/scrapy scrapy

docker run -it -v //D/Programs/scrapy:/root/scrapy scrapy

docker run -it -v scrapy:/root/scrapy scrapy

UPDATE WITH cp command based on @Makariy's feedback

docker images -a gives:

REPOSITORY                             TAG        IMAGE ID       CREATED        SIZE
scrapy                                 latest     e35e03c8cbbd   29 hours ago   5.71GB
<none>                                 <none>     2089ad178feb   29 hours ago   5.71GB
<none>                                 <none>     6162a0bec2fc   29 hours ago   5.7GB
<none>                                 <none>     116a0c593544   29 hours ago   5.7GB
mcr.microsoft.com/windows/servercore   ltsc2019   d1724c2d9a84   5 weeks ago    5.7GB

I run docker run -it scrapy and then docker container ls which gives:

CONTAINER ID   IMAGE     COMMAND                    CREATED              STATUS              PORTS     NAMES
1fcda458a14c   scrapy    "c:\\windows\\system32…"   About a minute ago   Up About a minute             thirsty_bassi

If I run docker cp D:\Programs\scrapy scrapy:/root/scrapy I get:

Error: No such container:path: scrapy:\root

So in a separate PowerShell instance I then run docker cp D:\Programs\scrapy thirsty_bassi:/root/scrapy whichs show no output in PowerShell whatsoever, so I think it should've done something.

But then in my container instance when I goto /root/scrapy folder I only see the files that were already added when the image was built, not the new ones I wanted to add.

Also, I think I'm adding files to the container here, but is there no way to add it to the image instead? Without rebuilding the whole image?

UPDATE 2

My folder structure:

D:\Programs
    Dockerfile
    \image_addons
    Dockerfile
        \scrapy

PS D:\Programs>docker build . -t scrapybase

Successfully built 95676d084e28
Successfully tagged scrapybase:latest

PS D:\Programs\image_addons> docker build -t scrapy .

Step 2/2 : COPY scrapy to /root/scrapy COPY failed: file not found in build context or excluded by .dockerignore: stat to: file does not exist

Dockerfile A

FROM mcr.microsoft.com/windows/servercore:ltsc2019
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
WORKDIR /root/scrapy

Dockerfile B

FROM scrapybase
COPY scrapy to /root/scrapy

Answer 1

answer if you want it quick and dirty

docker run -it -v c:/programs/test:/root/test ubuntu:latest cat /root/test/myTestFile.txt

to update one file quickly: If you don't have to build your code (I don't know what language you are using) you can build some base image with the initial code and when you want to change only one file (again I'm assuming you don't need to compile your project again for that, otherwise if you do that is not possible to due the nature of compiled programming language):

FROM previous-version-image:latest
COPY myfile dest/to/file

then because your CMD and ENTRYPOINT are saved from the previous stages no need to declare them. (if you don't remember use docker history <docker-image-name> to view virtual dockerfile for image to this stage).

Notice though not to repetitively use this method or you'll get a very big image with many useless layers. Use this only for quick testing and debugging.

explanation

Usually people use it for frontend development on docker containers but the basic idea persists, you create the basic working image with the dependencies installed and the directory layout setup with the last Dockerfile command being the development server start command.

example:

Dockerfile:

# pull the base image
FROM node:slim

# set the working directory
WORKDIR /app

# add `/app/node_modules/.bin` to $PATH
ENV PATH /app/node_modules/.bin:$PATH

# copy dependencies files
COPY package.json ./

COPY package-lock.json ./

# install app dependencies
RUN npm install

# add app
COPY . ./

# start development server
CMD ["npm", "start"]

startup command:

docker run -it --rm \
-v ${PWD}:/app \       <mount current working directory in host to container in path /app>
-v /app/node_modules \ <or other dependency directory if exists>
-p 80:3000 \           <ports if needs exposing>
ps-container:dev

I'm not sure if that use case will 100% work for you because it needs the code to be mounted using bind-mount all the time and when needed to be exported will have to be exported as the image and the source code directory, on the other hand, it allows you to make quick changes without waiting for the image to be built each time you add something new and in the end build the final image that contains all that's needed.

more relatable example to question provided code:

As you can see there is a file on the host machine that contains some text

the command that uses bind-mount to have access to the file:

docker run -it -v c:/programs/test:/root/test ubuntu:latest cat /root/test/myTestFile.txt

hope you find something that works for you from what I've provided here.

thanks to this tutorial and this example for starting examples and information.

EDIT:

Let's say your original Dockerfile looks like this:

FROM python:latest
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD python /app/app.py

This will build your initial image on top of we'll add layers and change the python files. The next Dockerfile we'd use (let's call it Dockerfile.fix file) would copy the file we want to change instead of the ones already in the image

FROM previous-image-name
COPY app.py .

Now with after building with this Dockerfile the final image Dockerfile would look (sort of) like so:

FROM python:latest
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD python /app/app.py
FROM previous-image-name
COPY app.py .

And each time we'll want to change the file we'll use the second Dockerfile

Answer 2

You also can use docker cp, to manually copy files from your host to running container

docker cp ./path/to/file containername:/another/path

Docs

Answer 3

There's no way you can change a Docker image without (at least partially) rebuilding it. But you don't have to rebuild all of it, you can just rebuild the layer copying your scrapy content.

You can optimize your build to have two images:

• First image is your static image you don't want to rebuild each time. Let's call it scrapy-base.
• Second and final image is based on first image scrapy-base and will only exist for the purpose of copying your dynamic scrapy content

scrapy-base's Dockerfile:

FROM mcr.microsoft.com/windows/servercore:ltsc2019
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

RUN mkdir root
RUN cd root
WORKDIR /root
RUN mkdir scrapy

And build it like:

docker build -t scrapy-base .

This command only needs to be run once. You won't have to build this image if you only change the content of local scrapy folder. (as you can see, the build does not use it at all)

scrapy's Dockerfile:

FROM scrapy-base

COPY scrapy /root/scrapy

With build command:

docker build -t scrapy .

This second build command will re-use the previous static image and only copy content without having to rebuild the entire image. Even with lots of files it should be pretty quick. You don't need to have a running container.

Answer 4

For your scenario :

docker run -v D:/test:/root/test your-image

A lots of valuable details available in this thread