Objective: Currently creating a dockerfile using jupyter/base-notebook, and I want to make sure that my environment can run in Windows, macOS, and Linux.
Issue: In theory, docker should not have an issue running in any OS. However, I encountered a following permission denied error when I was running on Linux:
pwd
+ docker run -it --rm -p 8888:8888 -v /home/<username>/myDocker:/home/jovyan myDocker:latest
[I 2022-07-18 18:01:42.359 ServerApp] jupyterlab | extension was successfully linked.
[W 2022-07-18 18:01:42.363 NotebookApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2022-07-18 18:01:42.364 NotebookApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2022-07-18 18:01:42.364 NotebookApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2022-07-18 18:01:42.371 ServerApp] nbclassic | extension was successfully linked.
[W 2022-07-18 18:01:42.372 ServerApp] [Errno 13] Permission denied: '/home/jovyan/.local/share'
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/traitlets/traitlets.py", line 645, in get
value = obj._trait_values[self.name]
KeyError: 'runtime_dir'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/bin/jupyter-lab", line 10, in <module>
sys.exit(main())
File "/opt/conda/lib/python3.10/site-packages/jupyter_server/extension/application.py", line 584, in launch_instance
serverapp = cls.initialize_server(argv=args)
File "/opt/conda/lib/python3.10/site-packages/jupyter_server/extension/application.py", line 554, in initialize_server
serverapp.initialize(
File "/opt/conda/lib/python3.10/site-packages/traitlets/config/application.py", line 110, in inner
return method(app, *args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/jupyter_server/serverapp.py", line 2462, in initialize
self.init_configurables()
File "/opt/conda/lib/python3.10/site-packages/jupyter_server/serverapp.py", line 1865, in init_configurables
connection_dir=self.runtime_dir,
File "/opt/conda/lib/python3.10/site-packages/traitlets/traitlets.py", line 686, in __get__
return self.get(obj, cls)
File "/opt/conda/lib/python3.10/site-packages/traitlets/traitlets.py", line 648, in get
default = obj.trait_defaults(self.name)
File "/opt/conda/lib/python3.10/site-packages/traitlets/traitlets.py", line 1752, in trait_defaults
return self._get_trait_default_generator(names[0])(self)
File "/opt/conda/lib/python3.10/site-packages/jupyter_core/application.py", line 106, in _runtime_dir_default
ensure_dir_exists(rd, mode=0o700)
File "/opt/conda/lib/python3.10/site-packages/jupyter_core/utils/__init__.py", line 12, in ensure_dir_exists
os.makedirs(path, mode=mode)
File "/opt/conda/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/opt/conda/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/opt/conda/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/home/jovyan/.local/share'
Where the main culprit of this is that .local/share is not being generated upon running dockerfile when ran on Linux (Ubuntu and RedHat). Surprisingly, when I run this on my Windows (via WSL2 - Ubuntu) and on macOS, it does generate .local/share and does not get this error.
Setup: My dockerfile looks something like this:
FROM jupyter/base-notebook:latest
USER root
USER jovyan
RUN set -ve
USER jovyan
EXPOSE 8888
ENTRYPOINT jupyter lab --no-browser --allow-root
*Note: My dockerfile has more content, but commented out most of them for debugging purpose.
Commands that I am using to build & run above dockerfile is following:
docker build -f <path_to_dockerfile> -t myDocker:latest .
docker run -it --rm -p 8888:8888 -v $(pwd):/home/jovyan myDocker:latest
Even after testing with above minimalistic dockerfile, Windows version would still generate the .local/share/... while giving me the permission denied error on Linux.
What I've tried:
So far, I've tried manually generating .local/share, which would get another permission denined error since it needs an additional directories (.local/share/jupyter/runtime/...).
New error looks something like this:
# new error 1
[I 2022-07-18 18:38:01.260 ServerApp] Writing Jupyter server cookie secret to /home/jovyan/.local/share/jupyter/runtime/jupyter_cookie_secret
[E 2022-07-18 18:38:01.260 ServerApp] Failed to write cookie secret to /home/jovyan/.local/share/jupyter/runtime/jupyter_cookie_secret: [Errno 13] Permission denied: '/home/jovyan/.local/share/jupyter/runtime/jupyter_cookie_secret'
/opt/conda/lib/python3.10/site-packages/IPython/paths.py:70: UserWarning: IPython parent '/home/jovyan' is not a writable location, using a temp directory.
warn("IPython parent '{0}' is not a writable location,"
# new error 2
[E 2022-07-18 18:38:01.580 ServerApp] Failed to write server-info to /home/jovyan/.local/share/jupyter/runtime/jpserver-1.json: [Errno 13] Permission denied: '/home/jovyan/.local/share/jupyter/runtime/jpserver-1.json'
Generating jpserver-1.json manually will still yield a permission error.
Additionally, I've used ls -la to check the permission status of every directory I have, and they all have same permission setting.
I've also referenced post with similar issue and used chown like following:
chown -R jovyan:users ~/.local/share/jupyter
but this did not solve issue as permission denied error comes from non-existing directories/files.
Question: What would I need to do so that Linux would generate .local/share/... to prevent permission denied error? Preferably, I would like to avoid generating .local/share/... manually.
Thank you in advance for your help.
Small Update: I've tried manually creating all missing files, such as:
.local/share/jupyter/runtime/jpserver-#.jp
.local/share/jupyter/runtime/jupyter_cookie_secret
and provided additional permission using:
chmod -R 777 .local/share/
and ran this whole thing, and it started working. However, it doesn't perfectly work as in it's still not generating essential directory, such as .jupyter.
Furthermore, I'd like to avoid this method as # in jpserver-#.jp is randomly generated, thus I won't be able to automatically generate these essential directories.
My speculation is that it's a path issue where jupyter is automatically generating essential directory, such as .local/share, .jupyter, etc. into different location.
