0

I am trying to write a docker-compose.yml which is then used in a continuous integration pipeline, but should also be possible to use locally.

I naively tried this:

services:
  app:
    build:
      ...
    image: ${IMAGE_SERVER_URL:-}/image_name:${IMAGE_TAG:-latest}

in my gitlab-ci.yml I log into the image server:

echo ${IMAGE_SERVER_PASSWORD} | docker login -u ${IMAGE_SERVER_USERNAME} --password-stdin ${IMAGE_SERVER_URL}

and then I can do

docker-compose build --pull
docker-compose push

So what is the issue?

when the .devcontainer from vscode attempts to build the app I get ERROR: Invalid Reference Format which is of course due to the fact that

/image-name:latest

is not a valid image name. So the issue is the slash.

docker-compose does not accept ${IMAGE_SERVER_URL+/} currently (cf. docker docs) so that is out of the question. I could of course include the slash in the environment variable but I have the feeling that this is going to cause similar problems in other places.

Are there any best practices when it comes to formatting dynamic image names?

Felix B.
  • 905
  • 9
  • 23
  • Is there a reason you cannot set variable IMAGE_SERVER_URL locally? – Konrad Botor Aug 13 '20 at 10:11
  • @KonradBotor You mean simply append the slash locally? Sure that would work - just makes everything a bit more brittle I guess – Felix B. Aug 13 '20 at 10:16
  • That's one solution, but I was thinking of `export IMAGE_SERVER_URL=something; code .` – Konrad Botor Aug 13 '20 at 10:23
  • @KonradBotor took me a while to realize that you actually meant `code` as in vscode and not a placeholder for more code. Yeah I guess you could set the environment variable localy. But the error message is not very descriptive and other devs would run into this problem too if they would pull the project. And I am trying to build a template here :-p – Felix B. Aug 13 '20 at 10:35

2 Answers2

1

There are two solutions that come to mind that satisfy OP's requirements.

One is to set the default value of IMAGE_SERVER_URL in the docker-compose.yml to a non-empty string (like OP did with IMAGE_TAG).

The other is to set said default value in the .env file placed alongside the docker-compose.yml.

As long as developers' are not meant to push their locally built image to the repo, it shouldn't matter what that default value is as long as it's not empty.

In both cases, according to the documentation, variables set in the shell by the CI server should override default values.

Edit: One other solution, less elegant in my opinion, is to use ${IMAGE_SERVER_URL:?Variable IMAGE_SERVER_URL must be set to a non-empty string} in docker-compose.yml to generate more comprehensible error message.

Konrad Botor
  • 4,765
  • 1
  • 16
  • 26
  • do you know anything about limits on file structure depth for images? I.e. am I allowed to have `folder1/folder2/..../foldern/image_name:tag` arbitrarily? Since this might have an impact later on if I have a deeper naming structure which might then break a default `IMAGE_SERVER_URL` if URLs are handled differently to folders. I can not find anything about the `valid reference format` which is annoying since the error `invalid reference format` obviously exists – Felix B. Aug 13 '20 at 11:03
  • Here's how official Docker registry implementation implements said reference format: https://github.com/docker/distribution/blob/master/reference/reference.go. If I understand correctly, you can have `server_url/path/path/name:tag` at most, but I guess it's possbile there are other registry implementations that support longer paths. you may want to ask a separate question about that. – Konrad Botor Aug 13 '20 at 11:18
  • So what happens if I state the image as `definitely_not_a_server_url/folder1/folder2/name:tag`? If it only prevents you from pushing the image, then this might even be desirable in some cases to prevent accidentally publishing private images – Felix B. Aug 13 '20 at 12:48
  • As far as I can tell the push will fail, but otherwise it should work fine. – Konrad Botor Aug 13 '20 at 12:57
  • @FelixB. a few registries support deeper nesting, but it's common to see only 2 layers, an organization or user, and then a specific image. The API itself doesn't restrict you from getting more complex, but various implementations may prevent you from defining a repository outside of their naming conventions. – BMitch Aug 13 '20 at 12:58
1

You can default your server url to Docker Hub's common name docker.io. And if you are using an official image from the library, you can also include library as the repository name:

${IMAGE_SERVER_URL:-docker.io}/image_name:${IMAGE_TAG:-latest}
BMitch
  • 231,797
  • 42
  • 475
  • 450
  • How is that any different to my idea of giving IMAGE_SERVER_URL a default value? – Konrad Botor Aug 13 '20 at 11:23
  • @KonradBotor I'm providing the default value for docker hub, which matches the expected behavior when no url is provided for docker users. I thought it was more useful to tell somewhat what string should be the default rather than just telling them to put a string there. – BMitch Aug 13 '20 at 12:29
  • @BMitch cf. my last question on Konrads answer: what do you mean by default? Is this default prepended to any image format where the root is not a server url? Does it check whether the root is a server url in some way? Does this mechanism prevent me from having `definitely_not_a_server_url/folder1/folder2/name:tag` images? – Felix B. Aug 13 '20 at 12:50
  • @FelixB. `debian:latest` and `docker.io/library/debian:latest` is the same image in docker. For docker to parse the first part of the path as a hostname, it needs to include a `:` or `.` inside the string before the first `/`. – BMitch Aug 13 '20 at 12:53
  • it is probably problematic to use `docker.io` as the root for private images which are not meant to be public (1. because it might accidentally get pushed although that is probably unlikely given that likely few people have the rights to push to docker.io/ibrary, 2. it might confuse the image with a different public image when caching for pulls) - both are not very likely I guess, but if you do not want to publish your images, `local.only/...` might be a better default. Don't you think? – Felix B. Aug 13 '20 at 13:44
  • @FelixB. If you are specifying an image without a registry server, then the docker engine assumes you are talking to docker hub. By private I assumed you were referencing a private repository. If you want to host your images elsewhere, then specify that host. If you don't want your images on any registry, then don't push and using something like `localhost:5000` for the registry may also help. – BMitch Aug 13 '20 at 13:48