8

What is the best practice for handling uid/gid and permissions with jupyter notebooks in docker?

  1. When one of the jupyter+python Dockerfiles in jupyter/docker-stack is run, a notebook gets saved with uid/gid 1000:100. This will fail if a mounted host folder is not writable by "other", which is an ugly approach.

  2. The notebook image can be run specifying the NB_UID and NB_GID, like this:

     docker run -p 8888:8888 -it --rm \
   -e NB_UID=$(id -u) \
   -e NB_GID=$(id -g) \
   -e GRANT_SUDO=yes \
   --user root \
   --mount type=bind,source="$(pwd)",target=/home/jovyan/work \
   myimage

    In this case, the uid/gid of joyvan in the container match my uid/gid, so there is no permissions problem writing to a mounted folder. However, now jovyan (the container user) cannot access /opt/conda, which is owned by 1000:100 and is not readable by other. So all the add-on packages cannot be loaded!

  3. We could also run docker build with --build-arg myuid=$(id -u) --build-arg mygid=$(id -g)

    I believe this would result in both /home/jovyan and /opt/conda being owned by the same uid:gid as me, everything good. However, the resulting image can be used only by me. If I give it to my collaborators (who has a different UID), it will not work.

So it seems that every possibility is blocked or a poor choice. File permissions in docker are difficult.

Can anyone share the best approach for this problem?

iBug
  • 35,554
  • 7
  • 89
  • 134
matchingmoments
  • 2,065
  • 2
  • 9
  • 7
  • You're mixing build-time with run-time, so by setting the `NB_UID` and `NB_GID` environment variables at run time (with the `docker run`) you're not changing the permissions that have been set during the build-time. Are the problems with the permissions happening on build time? In that case you don't need `RUN sudo` you can just say `USER root` then `RUN chown -R ...` and then `USER $NB_UID` again. If you need sudo during run time to work you could add the user to the sudoers and it'll work as well. – vstm Jul 01 '18 at 07:42
  • you are right, that is part of my confusion. I am going to edit the question to clarify. – matchingmoments Jul 02 '18 at 04:45

2 Answers2

10

The best practise with Jupyter Notebook is to use your own user id and group id so the new files you create will have correct ownership. Then use --group-add users to add yourself to users group to get access to the required folders (e.g. /opt/conda).

The full command would be:

docker run -it --rm --user $(id -u):$(id -g) --group-add users -v "$(pwd)":/home/jovyan -p 8888:8888 jupyter/scipy-notebook

Samuli Asmala
  • 1,755
  • 18
  • 24
1

I encountered the same problem and found a good solution which is referred from here.

COPY --chown=1000:100 hostfolder/* /home/$NB_USER/work/

Note that environment or argument expansion in command options is not implemented yet, thus following line would cause build error failed to build: unable to convert uid/gid chown string to host mapping: can't find uid for user $NB_UID: no such user: $NB_UID

# COPY --chown=$NB_USER:$NB_GID hostfolder/* /home/$NB_USER/work/

Therefore, need to hard code the user(jovyan) and group name(users) or id(1000:100).

lifang
  • 1,485
  • 3
  • 16
  • 23