6

Summary

So I'm trying to figure out a way to use docker to be able to spin up testing environments for customers rather easily. Basically, I've got a customized piece of software that want to install to a Windows docker container (microsoft/windowsservercore), and I need to be able to access the program folder for that software (C:\Program Files\SOFTWARE_NAME) as it has some logs, imports/exports, and other miscellaneous configuration files. The installation part was easy, and I figured that after a few hours of messing around with docker and learning how it works, but transferring files in a simple manner is proving far more difficult than I would expect. I'm well aware of the docker cp command, but I'd like something that allows for the files to be viewed in a file browser to allow testers to quickly/easily view log/configuration files from the container.

Background (what I've tried):

I've spent 20+ hours monkeying around with running an SSH server on the docker container, so I could just ssh in and move files back and forth, but I've had no luck. I've spent most of my time trying to configure OpenSSH, and I can get it installed, but there appears to be something wrong with the default configuration file provided with my installation, as I can't get it up and running unless I start it manually via command line by running sshd -d. Strangely, this runs just fine, but it isn't really a viable solution as it is running in debug mode and shuts down as soon as the connection is closed. I can provide more detail on what I've tested with this, but it seems like it might be a dead end (even though I feel like this should be extremely simple). I've followed every guide I can find (though half are specific to linux containers), and haven't gotten any of them to work, and half the posts I've found just say "why would you want to use ssh when you can just use the built in docker commands". I want to use ssh because it's simpler from an end user's perspective, and I'd rather tell a tester to ssh to a particular IP than make them interact with docker via the command line.

EDIT: Using OpenSSH

Starting server using net start sshd, which reports it starting successfully, however, the service stops immediately if I haven't generated at least an RSA or DSA key using:

ssh-keygen.exe -f "C:\\Program Files\\OpenSSH-Win64/./ssh_host_rsa_key" -t rsa

And modifying the permissions using:

icacls "C:\Program Files\OpenSSH-Win64/" /grant sshd:(OI)(CI)F /T

and

icacls "C:\Program Files\OpenSSH-Win64/" /grant ContainerAdministrator:(OI)(CI)F /T

Again, I'm using the default supplied sshd_config file, but I've tried just about every adjustment of those settings I can find and none of them help.

I also attempted to setup Volumes to do this, but because the installation of our software is done at compile time in docker, the folder that I want to map as a volume is already populated with files, which seems to make docker fail when I try to start the container with the volume attached. This section of documentation seems to say this should be possible, but I can't get it to work. Keep getting errors when I try to start the container saying "the directory is not empty".

EDIT: Command used: 

docker run -it -d -p 9999:9092 --mount source=my_volume,destination=C:/temp my_container

Running this on a ProxMox VM.

At this point, I'm running out of ideas, and something that I feel like should be incredibly simple is taking me far too many hours to figure out. It particularly frustrates me that I see so many blog posts saying "Just use the built in docker cp command!" when that is honestly a pretty bad solution when you're going to be browsing lots of files and viewing/editing them. I really need a method that allows the files to be viewed in a file browser/notepad++.

Is there something obvious here that I'm missing? How is this so difficult? Any help is appreciated.

Nealon
  • 2,213
  • 6
  • 26
  • 40
  • Could you post the exact docker command that you've used to mount the volume, please? There is a restriction on which order the options should be in for it to work, I've learned this the hard way. – Prav Aug 09 '18 at 20:00
  • Also, is your Docker runs on a virtual machine (VirtualBox/VMware)? – Prav Aug 09 '18 at 20:04
  • Could you please tell us exactly what SSH server you are using in your Windows docker container, and how exactly you have tried to start it (when not starting it manually via command line)? – Binarus Aug 09 '18 at 20:05
  • Post updated with the requested info @PraveenP Let me know if you need anything else. – Nealon Aug 09 '18 at 20:17
  • Post updated with the requested info @Binarus Let me know if you need anything else. – Nealon Aug 09 '18 at 20:17
  • So you're trying to mount Linux file system on an MS-DOS container? – Prav Aug 09 '18 at 20:37
  • Could you try this instead `docker run -v \`my_volume\`:\`C:/temp\` -it -d -p 9999:9092 my_container` – Prav Aug 09 '18 at 20:43
  • @PraveenP I get the error: docker: Error response from daemon: invalid volume specification: '`new_volume`:`C:/temp`'. – Nealon Aug 10 '18 at 12:53
  • Are you giving the path to `new_volume` as an absolute path? Relative path might not work – Prav Aug 10 '18 at 17:04
  • @PraveenP Aren't I referencing the volume by name here? Not path? Also, I thought that you could declare a new volume when starting the container? I could also point to an empty one, but I'd rather just do it all in one step. – Nealon Aug 10 '18 at 17:24

2 Answers2

2

So after a fair bit more troubleshooting, I was unable to get the docker volume to initialize on an already populated folder, even though the documentation suggests it should be possible.

So, I instead decided to try to start the container with the volume linked to an empty folder, and then start the installation script for the program after the container is running, so the folder populates after the volume is already linked. This worked perfectly! There's a bit of weirdness if you leave the files in the volume and then try to restart the container, as it will overwrite most of the files, but things like logs and files not created by the installer will remain, so we'll have to figure out some process for managing that, but it works just like I need it to, and then I can use windows sharing to access that volume folder from anywhere on the network.

Here's how I got it working, it's actually very simple.

So in my dockerfile, I added a batch script that unzips the installation DVD that is copied to the container, and runs the installer after extracting. I then used the CMD option to run this on container start:

Dockerfile

FROM microsoft/windowsservercore

ADD DVD.zip C:\\resources\\DVD.zip
ADD config.bat C:\\resources\\config.bat

CMD "C:\resources\config.bat" && cmd

Then I build the container without anything special:

docker build -t my_container:latest .

And run it with the attachment to the volume:

docker run -it -d -p 9999:9092 --mount source=my_volume,destination="C:/Program Files (x86)/{PROGRAM NAME}" my_container

And that's it. Unfortunately, the container takes a little longer to start (it does build faster though, for what that's worth, as it isn't running the installer in the build), and the program isn't installed/running for another 5 minutes or so after the container does start, but it works!

I can provide more details if anyone needs them, but most of the rest is implementation specific and fairly straightforward.

Nealon
  • 2,213
  • 6
  • 26
  • 40
1

Try this with Docker composer. Unfortunately, I cannot test it as I'm using a Mac it's not a "supported platform" (way to go Windows). See if that works, if not try volume line like this instead - ./my_volume:C:/tmp/

Dockerfile

FROM microsoft/windowsservercore

# need to ecape \
WORKDIR C:\\tmp\\

# Add the program from host machine to container
ADD ["<source>", "C:\tmp"]

# Normally used with web servers
# EXPOSE 80

# Running the program
CMD ["C:\tmp\program.exe", "any-parameter"]

Docker Composer

Should ideally be in the parent folder.

version: "3"

services:
  windows:
    build: ./folder-of-Dockerfile
    volume:
      - type: bind
        source: ./my_volume
        target: C:/tmp/

    ports:
      - 9999:9092

Folder structure

|---docker-composer.yml
    |
    |---folder-of-Dockerfile
        |
        |---Dockerfile

Just run docker-composer up to build and start the container. Use -d for detach mode, should only use once you know its working properly.

Useful link Manage Windows Dockerfile

Prav
  • 2,785
  • 1
  • 21
  • 30
  • I'm getting an error: `ERROR: for windows Cannot create container for service windows: invalid volume specification: 'C:\docker\my_volume:/c/temp/:rw'` I tried making sure I had declared the volume using `docker volume create my_volume` before, but that doesn't seem to help. – Nealon Aug 13 '18 at 13:32
  • It could be just `/c/` that Unix substitution of Windows file paths. You can try `'C:\'` instead to see if that works. – Prav Aug 13 '18 at 17:16
  • That seems to have worked... thanks for that, but either I'm misunderstanding the use of Docker compose, or it's doing something strange. I get a `old_windows_1 exited with code 0` message before the command prompt returns to allow user input, and when I run `docker container ls`, the container doesn't show as running. Also, the volume folder on the VM is empty. – Nealon Aug 13 '18 at 17:21
  • So how Docker run is that it require a foreground process to determine whether to keep the container up or not. This can be as simple as never-ending for loop. Do you have any foreground process in the container? For Docker, there's no background only process. Although you can have them as long as there's at least one foreground process. – Prav Aug 13 '18 at 17:26
  • The application I install in the dockerfile has 2 different services that run on the container. Not sure if those are technically "foreground" but when I start the dockerfile normally using `docker run`, the container remains running after I start it. – Nealon Aug 13 '18 at 17:54
  • I think I know what's going on. You need to build an image from the image above, during the build you need to define an application that run on foreground. Look in to `Dockerfile` if you haven't already. – Prav Aug 13 '18 at 18:03
  • So in the dockerfile that this docker compose file is using, I set the image to: `FROM microsoft/windowsservercore`. Are you saying that the docker compose file should reference a different image then? – Nealon Aug 13 '18 at 18:17
  • Yes that's correct, as `microsoft/windowsservercore` is a base image and you need to build your application from this image. Like baking a cake with layers. – Prav Aug 13 '18 at 18:25
  • Thanks, that helped to clarify. However, now I'm getting that same error that I was having before when just using the dockerfile, that "The directory is not empty". It really just seems like this functionality isn't supported... even though the documentation says it is... – Nealon Aug 13 '18 at 19:21
  • Try moving the `ADD` destination path to somewhere else and update that `CMD` line as well. – Prav Aug 13 '18 at 19:23
  • Or add `VOLUME` to Dockerfile and remove form composer – Prav Aug 13 '18 at 19:26
  • Also I think there's restriction on `C:\` path for volume as well. https://docs.docker.com/engine/reference/builder/#notes-about-specifying-volumes – Prav Aug 13 '18 at 19:31
  • Thanks, I'm giving this a shot now... having a couple small unrelated issues with HNS errors, but I'll deal with those and let you know the results. Your help is much appreciated. – Nealon Aug 13 '18 at 20:01
  • 1
    Just as an update, still dealing with those HNS errors. I think there might be something wrong with the VM I'm using at this point... I'll update when I'm able to test moving the volume to the D drive. – Nealon Aug 14 '18 at 19:32
  • Another update... Some other stuff came up that I had to work on, and I'm just now getting around to setting up a fresh VM to test this on, but I probably won't have the chance today. I'm out of town for the weekend, so I'll plan on resuming this monday. I'll probably have to redo the bounty (? Not sure haven't done a bounty before this), and if your answer ends up working, I'll be sure to accept yours and make sure you get the credit. Sorry for the delays, and thank you very much for your help. – Nealon Aug 16 '18 at 19:11
  • No worries, it's important that your problem get solve, so let me know if it throws any new problems – Prav Aug 16 '18 at 19:32
  • Just got around to testing again... still getting the "directory is not empty" error when using the D drive. – Nealon Aug 20 '18 at 20:55
  • I've updated the `volume:` block. Can you check if that works? – Prav Aug 22 '18 at 16:20
  • same thing :( "directory is not empty". – Nealon Aug 22 '18 at 17:54
  • I'm currently looking into the possibility of running the installation after the container is running via some script. That would allow me to link the volume to an empty folder and then populate it afterwards. – Nealon Aug 22 '18 at 17:55
  • 1
    Just FYI, I got the solution I describe above to work, and posted my own answer describing it in more detail. Thank you very much for all of your troubleshooting help with all of this! – Nealon Aug 23 '18 at 15:13