1

I'm trying to copy my entire home directory or at least make it available to my docker build, but I get the following error:

error from sender: open .Trash: operation not permitted

These are the instructions I was writing for myself:

Run docker build from ~/username (which makes everything starting at username available) e.g.:

/Users/miranda9

then run:

docker build -t test:latest -f py_repo/pycoq_dockerfile/Dockerfile .

But I get the error I mentioned above. Is there a way around this?

My attempt is based on this answer.

Current docker file:

# FROM ubuntu:20.04
FROM ubuntu:18.04

# after this cmd RUN pwd should print /miranda9
WORKDIR /miranda9

# The COPY instruction copies new files or directories from <src> and adds them to the filesystem of the container at the path <dest>.
COPY . .

I'd also prefer to not have to remake the image each time. e.g. just running my container as if it was a command would be best. But I am also curious to see a solution that requires rebuilding the image.

Related question I asked many years ago: How to use a python library that is constantly changing in a docker image or new container?

Attempt 2: solution without a docker file but using mounts volumes

I wanted to avoid entirely using a docker file but I couldn't get that working:

# docker run -v /Users/miranda9:/home/miranda9 -ti continuumio/miniconda3 bash
# docker run -v /Users/miranda9:/ -ti continuumio/miniconda3 bash

gives error:

docker: Error response from daemon: invalid volume specification: '/host_mnt/Users/miranda9:/': invalid mount config for type "bind": invalid specification: destination can't be '/'.
Charlie Parker
  • 5,884
  • 57
  • 198
  • 323
  • 1
    @LMC will post it tomorrow. Lost access to the code. But its very simple. Just `FROM ubuntu:18.04` and `COPY . .` for now as an initial attempt. Will update tomorrow. – Charlie Parker Aug 18 '21 at 23:08
  • 2
    Typically you'd want to set the build context directory to something as close to the application as you can. Internally Docker creates a tar file of the entire context directory and sends it to itself over a socket, so something like your home directory isn't a great choice. Can you `docker build py_repo/pycoq_dockerfile`, without a `-f` option? – David Maze Aug 19 '21 at 00:53
  • 2
    I guess the `.Trash` is a special system directory with special permissions. So you don't have access to copy it. Not sure why you want to copy exactly entire user directory, but it doesn't look like a good idea. Also you can use array to copy exact files if it possible: `COPY ["source_folder1", "source_file1", "./destination_folder"]`. That will help to avoid to copy system files and folders. – rzlvmp Aug 19 '21 at 01:19
  • @rzlvmp I have multiple repos of different projects that interact with each other. e.g. I have a `ultimate-utils` repo that my main folder uses and `pycoq` and my main repo `main-repo` and all are in either development mode (e.g. either `conda develop .` or `pip -e install .`). So whenever I run my code I want the docker container to just have everything. I guess the easiest hack would be to have a special folder inside my home `miranda/projects` where I have all my projects there so that the `.Trash` and nonthing else special is there for me to work...not sure what else is a good idea... – Charlie Parker Aug 19 '21 at 17:02
  • I'd also prefer to not have to remake the image each time. e.g. just running my container as if it was a command would be best. – Charlie Parker Aug 19 '21 at 17:05
  • btw I am also curious to see a solution that requires rebuilding the image – Charlie Parker Aug 19 '21 at 21:28
  • out of curiosity, if I mount all those files if I edit the mounted files or create new files in the container in the dir tha thas been mounted - will they appear outside of the mount i.e. in the original system as the container runs? – Charlie Parker Aug 20 '21 at 16:23
  • try with a slash after the path `docker run -v /Users/miranda9/:/home/miranda9/ -ti continuumio/miniconda3 bash` – flaxon Aug 24 '21 at 22:35

3 Answers3

1

Since you've mentioned not wanting to remake the image each time, would it be sufficient to just volume map your home directory?

So remove the WORKDIR and COPY directives and just map the volume when you run the docker?

e.g.

docker run -v /Users/miranda9:/path/in/container

pahool
  • 316
  • 2
  • 16
  • will try it out! btw why would that work but not the `COPY`? Is it because `COPY` needs remaking the image? Btw, does the path to the folder `/path/in/container` has to exist in the container? (btw if you have minimal image I can build and try that would be fantastic and would def accept your answer) – Charlie Parker Aug 19 '21 at 19:27
  • hmmm why didn't this work? ` docker run -v /Users/miranda9:/ -ti continuumio/miniconda3 bash` gave me error `docker: Error response from daemon: invalid volume specification: '/host_mnt/Users/miranda9:/': invalid mount config for type "bind": invalid specification: destination can't be '/'.`? – Charlie Parker Aug 19 '21 at 20:09
  • btw if I write things into my docker container that is now mounted to my real home directory...if I edit files or do install etc, will it modify the contents of my original files? or add folders files etc if I do installs or do things in general? – Charlie Parker Aug 19 '21 at 21:23
  • You can't mount the host volume at the docker root directory. This would overwrite the entire docker filesystem. Try mounting at /opt or /miranda9 – pahool Aug 20 '21 at 16:29
  • what would be an option that doesn't require me to write a `Dockerfile`? – Charlie Parker Aug 20 '21 at 17:30
  • 1
    and answering one of your questions above. Yes, if you modify the files on the bind mount inside the docker, those changes will propagate to the host filesystem. Is that the behavior you want? Because there are other options for mounting volumes. – pahool Aug 20 '21 at 17:32
  • What is it exactly that you want the docker to do? Are you going to be getting a shell in it? Is the vanilla ubuntu:20.04 sufficient for whatever task you are trying to accomplish? – pahool Aug 20 '21 at 17:34
  • I think I do want to be able to modify the original file system from Docker which is what you suggested. But I am also curious to know what other mounting volume options exist that are common use. Perhaps something that seems useful is a mount that is like a `COPY`? or not sure what other would be a useful use. – Charlie Parker Aug 20 '21 at 17:34
  • I use the `python` or `docker run -ti continuumio/miniconda3` since it has python already in it. – Charlie Parker Aug 20 '21 at 17:35
  • For example, you could mount the volume read only, so that the docker couldn't modify the filesystem. I believe you could also create an overlay filesystem so that changes made would persist for the life of the docker but wouldn't persist on the host filesystem. I'm not sure how to do that, but I'm pretty sure it's do-able. – pahool Aug 20 '21 at 17:36
  • I see, so you're basically using it to get python. There may be better images for that (https://hub.docker.com/_/python/), but, using the ubuntu image, you wouldn't need to create a docker image using a Dockerfile. Just run: `docker run -it -v /Users/miranda9:/opt ubuntu:18.04 bash` – pahool Aug 20 '21 at 17:41
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/236243/discussion-between-pahool-and-charlie-parker). – pahool Aug 20 '21 at 17:42
0

I think this works by mounting my local dir:

Use this dockerfile:

FROM ubuntu:20.04
FROM python:latest
FROM continuumio/miniconda3

MAINTAINER Miranda "miranda9_fake_email@gmail.com"

# create this dir in the docker container
RUN mkdir -p /home/miranda9/
# make it the working dir
WORKDIR /home/miranda9

then build the image from that:

docker build -t miranda9_playground .

then run the docker container and mount the home directory to the home directory in the docker container:

docker run -v /Users/miranda9:/home/miranda9 -ti miranda9_playground bash

this last -v for mounting the <local_dir>:<docker_dir> the the -ti is -t and -i one for tagging the container/application the other for interactive mode -i. Then the name of the image to build the container from miranda9_playground and the final one the shell (the command) to have docker run inside that container in this case bash.

Docker container in this repo: https://github.com/brando90/ultimate-utils

I wanted to avoid entirely using a docker file but I couldn't get that working:

# docker run -v /Users/miranda9:/home/miranda9 -ti continuumio/miniconda3 bash
# docker run -v /Users/miranda9:/ -ti continuumio/miniconda3 bash

gives error:

docker: Error response from daemon: invalid volume specification: '/host_mnt/Users/miranda9:/': invalid mount config for type "bind": invalid specification: destination can't be '/'.
Charlie Parker
  • 5,884
  • 57
  • 198
  • 323
  • `invalid specification: destination can't be '/'.` → just run `docker run -v /Users/miranda9:/app_root -w /app_root -ti continuumio/miniconda3 bash` https://docs.docker.com/engine/reference/run/#workdir – rzlvmp Aug 20 '21 at 00:39
0

This works for me on wsl2

/mnt$ docker run -v ~/:/home/miranda -ti continuumio/miniconda3 bash
(base) root@d7ef57cffe53:/# ls /home/miranda/
BSD  go  projects
flaxon
  • 928
  • 4
  • 18