What is a best-practice to provide custom command line tools for your containerized application?

Question

we are a software vendor that is currently investigating Docker in general and Kubernetes in particular as a way to allow customers to install our software stack in their environment. Our software comes with a whole bunch of command line tools, which are mainly used for adminstration purposes. While most admin tasks can be done through the UI, others can only be done through these tools. We thought on ways to provide these tools to customers running our software on Docker. First thought was to just make a big archive available to them, which they can download and run on their admin machine. But we quickly came up with the idea to instead provide an "admin tools" container that contains all these tools. Different from our other containers, this one is not meant to be run as a server, but to be run either from scripts or interactively. Even thought the corresponding docker or kubectl command lines get a bit lengthy, this mostly works and I think the approach has quite some merit, as it is an "in-band" way to also publish our command line tools. However, some administration tasks require you to pass files to the respective command line tool or the tools generate files (e.g., backups). So you need a way to get these files into or out of your container. In a Docker environment, it is quite straightforward to use "docker run" and mount a volume or host directory into your container which contains the files. In Kubernetes, this seem to be not as straightforward, though (see Create kubernetes pod with volume using kubectl run for an approach to do this with "kubectl run"... Yikes!). Running the admin tools container in a regular pod (created with a YAML file) with a volume mount and attaching to it is actually simpler.

In the end, I would like to get your thoughts on the title question: What is the "best" way to make command line tools for the administration of containerized applications available to their users?

• Ship them as a dedicated archive and let users use them as they did in the pre-container world?
• Ship them in containers that are meant to be used interactively (as described above)?
• any other ideas?

Regards PJ

Answer 1

You want Kubernetes Operators.

People don't want to use your CLI, they are only using your CLI to "get stuff done". If you can put your CLI actions into a Kubernetes Operator, you get a standard API that any other tool can use.

For example, let's say our software is kinda like a database. Your operator could have an entry representing each database, which would have fields for "how often to backup" and "how many replicas". Any tool (or even the kubctl CLI) can set those fields, and your operator does all the dirty work to make it happen. Your customers install a single operator, and they don't have to manage the transient containers that "get work done", nor do they have to understand your CLI.

Answer 2

The following options should be available to users:

• The tools should be available as docker images with tags
• Users should be free to use docker images either with docker run or kubernetes according to what works for them ( leave it to users )
• The tools should also be available as tar archives as in pre docker world , because not all the users would like to use docker
• The tools can also be provided/hosted on the cloud and users can run them from there. see https://cmd.io/

Answer 3

I like command-line tools as much as the next software engineer; and my opinion is that Docker should not be a part of your solution.

As a tool consumer, I'm pretty conscious of the two important details that I need root-equivalent privileges to run something in Docker at all, and that Docker containers have separate filesystem spaces from my main application. Using Docker itself as an example, it's good to know that I can

docker images | grep dmaze | awk '{ print $1 ":" $2 }' | xargs docker rmi

but that becomes a lot harder to do if I replace docker with an extended docker run -e ... -v ... -v ... vendor/imagename command invocation.

Similarly, it's worth browsing Stack Overflow for questions like this:

I've packaged my tool in Docker. I'm trying to run it. What docker run -v option do I need to give it access to my files? It doesn't run as root; what permission settings do I need? If I have one file that references another, how do I make the filenames consistent across environments? I forgot to docker run -v, is my output file totally gone now?

If your tools are packaged as a Docker image, you're pushing all of these complexities off on to the user; just dropping something in $PATH is much simpler to me to use.

The advantages to you as a tool vendor seem pretty minimal. The Docker image is a single artifact, but so is a tar file. You can probably rely on a common set of base C libraries and language runtimes being readily available; you can get a lot done with the standard Python library, for example, and almost every Linux installation will have that available.

Kubernetes is definitely not applicable here. It's most useful for running distributed applications in cluster environments. Some very dedicated developers might have minikube available, but it's not an every-day thing, and getting an interactive shell in a Kubernetes pod is even more of an exception than getting one in a Docker container is (or is supposed to be).