A working how-to for data extraction of non-root named volume permissions working with linux and win

Question

I'm trying a simple workflow without success and it take me a loooooot of time to test many solutions on SO and github. Permission for named folder and more generaly permissions volume in docker is a nightmare link1 link2 imho.

So i restart from scratch, trying to create a simple proof of concept for my use case.

I want this general workflow :

user on windows and/or linux build the Dockerfile
user run the container (if possible not as root)
the container launch a crontab which run a script writing in the data volume each minute
users (on linux or windows) get the results from the data volume (not root) because permissions are correctly mapped

I use supercronic because it runs crontab in container without root permission.

The Dockerfile :

FROM artemklevtsov/r-alpine:latest as baseImage

RUN mkdir -p /usr/local/src/myscript/
RUN mkdir -p /usr/local/src/myscript/result

COPY . /usr/local/src/myscript/

WORKDIR /usr/local/src/myscript/

RUN echo http://nl.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories
RUN apk --no-cache add busybox-suid curl

ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.$
    SUPERCRONIC=supercronic-linux-amd64 \
    SUPERCRONIC_SHA1SUM=9aeb41e00cc7b71d30d33c57a2333f2c2581a201

RUN curl -fsSLO "$SUPERCRONIC_URL" \
 && echo "${SUPERCRONIC_SHA1SUM}  ${SUPERCRONIC}" | sha1sum -c - \
 && chmod +x "$SUPERCRONIC" \
 && mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
 && ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic

CMD ["supercronic", "crontab"]

The crontab file :

* * * * * sh /usr/local/src/myscript/run.sh > /proc/1/fd/1 2>&1

The run.sh script

#!/bin/bash

name=$(date '+%Y-%m-%d-%s')

echo "some data for the file" >> ./result/fileName$name

The commands :

# create the volume for result, uid/gid option are not possible for windows 
docker volume create --name myTestVolume

docker run --mount type=volume,source=myTestVolume,destination=/usr/local/src/myscript/result test

docker run --rm -v myTestVolume:/alpine_data -v $(pwd)/local_backup:/alpine_backup alpine:latest tar cvf /alpine_backup/scrap_data_"$(date '+%y-%m-%d')".tar /alpine_data

When i do this the result folder local_backup and files it contains has root:root permissions, so user who launch this container cannot access the files.

Is there a solution which works, which permits windows/linux/mac users who launch the same script to access easily the files into volume without problem of permissions ?

EDIT 1 :

The strategy first described here only work with binded volume, and not named volume. We use an entrypoint.sh to chown uid/gid of folders of container based on information given by docker run.

I copy paste the modified Dockerfile :

FROM artemklevtsov/r-alpine:latest as baseImage

RUN mkdir -p /usr/local/src/myscript/
RUN mkdir -p /usr/local/src/myscript/result

COPY . /usr/local/src/myscript/

ENTRYPOINT [ "/usr/local/src/myscript/entrypoint.sh" ]

WORKDIR /usr/local/src/myscript/

RUN echo http://nl.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories
RUN apk --no-cache add busybox-suid curl su-exec

ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.1.$
    SUPERCRONIC=supercronic-linux-amd64 \
    SUPERCRONIC_SHA1SUM=9aeb41e00cc7b71d30d33c57a2333f2c2581a201

RUN curl -fsSLO "$SUPERCRONIC_URL" \
 && echo "${SUPERCRONIC_SHA1SUM}  ${SUPERCRONIC}" | sha1sum -c - \
 && chmod +x "$SUPERCRONIC" \
 && mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
 && ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic

CMD ["supercronic", "crontab"]

The entrypoint.sh

#!/bin/sh
set -e

addgroup -g $GID scrap && adduser -s /bin/sh -D -G scrap -u $UID scrap

if [ "$(whoami)" == "root" ]; then
    chown -R scrap:scrap /usr/local/src/myscript/
    chown --dereference scrap "/proc/$$/fd/1" "/proc/$$/fd/2" || :
    exec su-exec scrap "$@"
fi

The procedure to build,launch, export:

docker build . --tag=test                                                       

docker run -e UID=1000 -e GID=1000 --mount type=volume,source=myTestVolume,destination=/usr/local/src/myscript/result test

 docker run --rm -v myTestVolume:/alpine_data -v $(pwd)/local_backup:/alpine_backup alpine:latest tar cvf /alpine_backup/scrap_data_"$(date '+%y-%m-%d')".tar /alpine_data

EDIT 2 :

For Windows, using docker toolbox and binded volume, i found the answer on SO. I use the c:/Users/MyUsers folder for binding, it's more simple.

docker run --name test -d -e UID=1000 -e GID=1000 --mount type=bind,source=/c/Users/myusers/localbackup,destination=/usr/local/src/myscript/result dockertest --name rflightscraps

Result of investigation

crontab run with scrap user [OK]
UID/GID of local user are mapped to container user scrap [OK]
Exported data continue to be root [NOT OK].
Windows / Linux [HALF OK]

If i use bind volume and not a named volume, it works. But this is not the desired behavior, how can i use the named volume with correct permission on Win/Linux ...

This seems like almost a trivial problem using the host's cron daemon, until you introduce Docker. What do you hope to gain by using Docker? — David Maze, Sep 20 '18 at 22:01
I'm using this use case for courses, each student run this worflow first on their local machine and after we push images to some servers. — reyman64, Sep 21 '18 at 06:55
You can't award me the bounty post grace period since [**the answer was not accepted at that time :-(**](https://meta.stackoverflow.com/q/332116/6663095) — Mani, Oct 11 '18 at 04:10
[This question is being discussed on meta.](https://meta.stackoverflow.com/questions/375208/bounty-should-be-awarded-to-accepted-answer-if-it-expires-failing-the-criterias) — Script47, Oct 11 '18 at 14:47

Mani · Accepted Answer · 2018-10-10T16:38:36.413

Let me divide the answer into two parts Linux Part and Docker part. You need to understand both in order to solve this problem.

Linux Part

It is easy to run cronjobs as user other than root in Linux.

This can be achieved by creating a user in docker container with the same UID as of that in the host machine and copying the crontab file as /var/spool/cron/crontabs/user_name.

From man crontab

crontab is the program used to install, deinstall or list the tables used to drive the cron(8) daemon in Vixie Cron. Each user can have their own crontab, and though these are files in /var/spool/cron/crontabs, they are not intended to be edited directly.

Since Linux identifies users by User Id, inside docker the UID will be bound to the newly created user whereas in host machine the same will be binded with host user.

So, You don't have any permission issue as the files is owned by the host_user. Now you would have understood why I mentioned creating user with same UID as of that in host machine.

Docker Part

Docker considers all the directories(or layers) to be UNION FILE SYSTEM. Whenever you build an image each instruction creates a layer and the layer is marked as read-only. This is the reason Docker containers doesn't persist data. So you have to explicitly tell docker that some directories need to persist data by using VOLUME keyword.

You can run containers without mentioning volume explicitly. If you do so, docker daemon considers them to be UFS and resets the permissions. In order to preserve the changes to a file/directory including ownership. The respective file should be declared as Volume in Dockerfile.

From UNION FILE SYSTEM

Indeed, when a container has booted, it is moved into memory, and the boot filesystem is unmounted to free up the RAM used by the initrd disk image. So far this looks pretty much like a typical Linux virtualization stack. Indeed, Docker next layers a root filesystem, rootfs, on top of the boot filesystem. This rootfs can be one or more operating systems (e.g., a Debian or Ubuntu filesystem). Docker calls each of these filesystems images. Images can be layered on top of one another. The image below is called the parent image and you can traverse each layer until you reach the bottom of the image stack where the final image is called the base image. Finally, when a container is launched from an image, Docker mounts a read-write filesystem on top of any layers below. This is where whatever processes we want our Docker container to run will execute. When Docker first starts a container, the initial read-write layer is empty. As changes occur, they are applied to this layer; for example, if you want to change a file, then that file will be copied from the read-only layer below into the read-write layer. The read-only version of the file will still exist but is now hidden underneath the copy.

Example:

Let us assume that we have a user called host_user. The UID of host_user is 1000. Now we are going to create a user called docker_user in Docker container. So I'll assign him UID as 1000. Now whatever files that are owned by docker_user in Docker container is also owned by host_user if those files are accessible by host_user from host(i.e through volumes).

Now you can share the binded directory with others without any permission issues. You can even give 777 permission on the corresponding directory which allows others to edit the data. Else, You can leave 755 permissions which allows others to copy but only the owner to edit the data.

I've declared the directory to persist changes as a volume. This preserves all changes. Be careful as once you declare a directory as volume further changes made to that directory while building the will be ignored as those changes will be in separate layers. Hence do all your changes in the directory and then declare it as volume.

Here is the Docker file.

FROM alpine:latest
ARG ID=1000 
#UID as arg so we can also pass custom user_id
ARG CRON_USER=docker_user
#same goes for username

COPY crontab /var/spool/cron/crontabs/$CRON_USER
RUN adduser -g "Custom Cron User" -DH -u $ID $CRON_USER && \
    chmod 0600 /var/spool/cron/crontabs/$CRON_USER && \
    mkdir /temp && \
    chown -R $ID:$ID /temp && \
    chmod 777 /temp

VOLUME /temp
#Specify the dir to be preserved as Volume else docker considers it as Union File System

ENTRYPOINT ["crond", "-f", "-l", "2"]

Here is the crontab

* * * * * /usr/bin/whoami >> /temp/cron.log

Building the image

docker build . -t test

Create new volume

docker volume create --name myTestVolume

Run with Data volume

docker run --rm --name test -d -v myTestVolume:/usr/local/src/myscript/result test:latest

Whenever you mount myTestVolume to other container you can see the data under /usr/local/src/myscript/result is owned by UID 1000 if no user exist with that UID in that container or the username of corresponding UID.

Run with Bind volume

docker run --rm --name test - -dv $PWD:/usr/local/src/myscript/result test:latest

When you do an ls -al /home/host_user/temp You will see that file called cron.log is created and is owned by **host_user**.

The same will be owned by docker_user in docker container when you do an ls -al /temp. The contents of cron.log will be docker_user.

So, Your effective Dockerfile should be

FROM artemklevtsov/r-alpine:latest as baseImage

ARG ID=1000 

ARG CRON_USER=docker_user

RUN adduser -g "Custom Cron User" -DH -u $ID $CRON_USER && \
    chmod 0600 /var/spool/cron/crontabs/$CRON_USER && \
    echo http://nl.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories && \
    apk --no-cache add busybox-suid curl && \
    mkdir -p /usr/local/src/myscript/result && \
    chown -R $ID:$ID /usr/local/src/myscript/result && \
    chmod 777 /usr/local/src/myscript/result

COPY crontab /var/spool/cron/crontabs/$CRON_USER

COPY . /usr/local/src/myscript/

VOLUME /usr/local/src/myscript/result
#This preserves chown and chmod changes.

WORKDIR /usr/local/src/myscript/

ENTRYPOINT ["crond", "-f", "-l", "2"]

Now whenever you attach a Data/bind volume to /usr/local/src/myscript/result it will be owned by user having UID 1000 and the same is persistent across all the containers whichever has mounted the same volume with their corresponding user with 1000 as file owners.

Please Note: I've given 777 permissions in order to share with every one. You can skip that step in your Dockerfle based on your convinence.

References:

Hum, you're right, it works, but you use binded volume (like say in edit of my question) and not the new "named volume" canonical way, like i search to solve in my question. — reyman64, Oct 09 '18 at 20:04
Thanks for your effort to solve this problem. I don't see the part where you create a [named volume](https://docs.docker.com/storage/volumes/) : `docker volume create --name myTestVolume` — reyman64, Oct 10 '18 at 08:05

A working how-to for data extraction of non-root named volume permissions working with linux and win

1 Answers1

Linux Part

Docker Part

Example: