docker-compose up not mounting volumes in the host directory

Question

I have Ubuntu 20.04 machine and my docker version is 20.10.08. Following is my docker-compose file:

services:
  zookeeper:
    image: docker.io/bitnami/zookeeper:3.7
    ports:
      - "2181:2181"
    volumes:
      - "kafka-vol:/bitnami/kafka/config"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: docker.io/bitnami/kafka:3
    ports:
      - "9092:9092"
    volumes:
      - "kafka-vol:/bitnami/kafka/config"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper
volumes:
  kafka-vol:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /home/ned/advice/data/conf/kafka

I am using docker-compose for running the containers. It is running fine with no errors but my configs of kafka are not mounting in the host directory. So far I have tried the following:

1. Allowed the permission 1001 on the folder as mentioned in the bitnami docker image documentation.
2. Gave relative path .advice/data/conf/kafka in the volumes
3. Gave full path direct in the volumes of the services as (without the double quotes as suggested in Docker Volume not mounting any files):

    image: docker.io/bitnami/kafka:3
    ports:
      - "9092:9092"
    volumes:
      - advice/data/conf/kafka:/bitnami/kafka/config

When I do docker volume inspect kafka-vol. It gave me the following output:

[
    {
        "CreatedAt": "2022-01-28T07:16:22Z",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "ned",
            "com.docker.compose.version": "2.2.3",
            "com.docker.compose.volume": "kafka-vol"
        },
        "Mountpoint": "/var/snap/docker/common/var-lib-docker/volumes/ned_kafka-vol/_data",
        "Name": "ned_kafka-vol",
        "Options": {
            "device": "/home/ned/advice/data/conf/kafka",
            "o": "bind",
            "type": "none"
        },
        "Scope": "local"
    }
]

Now, I don't know what am I doing wrong that it isn't mounting my config files. The directory /home/ned/advice/data/conf/kafka is empty. I was expecting docker's kafka config at this /home/ned/advice/data/conf/kafka directory

Answer 1

It looks like this directory in this container is meant to mount external config files, it's completely empty. It's also called KAFKA_MOUNTED_CONF_DIR in their env script.

So If you want to mount your own config, them probably like this.

mkdir config
echo foobar > config/config.txt
chown -R 1001:1001 config

  kafka:
    image: docker.io/bitnami/kafka:3
    ports:
      - "9092:9092"
    volumes:
      - "./config:/bitnami/kafka/config"

$ docker-compose up -d
$ docker exec -ti test_kafka_1 ls /bitnami/kafka/config
config.txt

As @OneCricketeer found out, if you want to get the default config it's using, you need to get that from /opt/bitnami/kafka/config.

If you want to mount the Kafka default configs to the host system, you need to use that path.

version: "2"

services:
  zookeeper:
    image: docker.io/bitnami/zookeeper:3.7
    ports:
      - "2181:2181"
    volumes:
      - "zookeeper_data:/bitnami"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: docker.io/bitnami/kafka:3
    ports:
      - "9092:9092"
    volumes:
      - "kafka_data:/bitnami"
      # the volume should be mounted to this path
      - "kafa_configs:/opt/bitnami/kafka/config"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper

volumes:
  zookeeper_data:
    driver: local
  kafka_data:
    driver: local
  # this is your volume
  kafa_configs:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: ./config

$ mkidr config
$ sudo chown 1001:1001 config
$ docker compose up -d
ls -ltrh config/
total 76K
-rw-rw-r-- 1 root root 1.2K Jan 24 23:14 zookeeper.properties
-rw-rw-r-- 1 root root 1.2K Jan 24 23:14 trogdor.conf
-rw-rw-r-- 1 root root 1.1K Jan 24 23:14 tools-log4j.properties
-rw-rw-r-- 1 root root 4.6K Jan 24 23:14 log4j.properties
-rw-rw-r-- 1 root root 2.3K Jan 24 23:14 connect-standalone.properties
-rw-rw-r-- 1 root root 2.5K Jan 24 23:14 connect-mirror-maker.properties
-rw-rw-r-- 1 root root 2.1K Jan 24 23:14 connect-log4j.properties
-rw-rw-r-- 1 root root  881 Jan 24 23:14 connect-file-source.properties
-rw-rw-r-- 1 root root  883 Jan 24 23:14 connect-file-sink.properties
-rw-rw-r-- 1 root root 5.4K Jan 24 23:14 connect-distributed.properties
-rw-rw-r-- 1 root root  909 Jan 24 23:14 connect-console-source.properties
-rw-rw-r-- 1 root root  906 Jan 24 23:14 connect-console-sink.properties
drwxrwxr-x 2 root root 4.0K Jan 29 17:05 kraft
-rw-rw-r-- 1 root root 6.9K Jan 29 17:05 server.properties
-rw-rw-r-- 1 root root 1.9K Jan 29 17:05 producer.properties
-rw-rw-r-- 1 root root 1.3K Jan 29 17:05 consumer.properties

Depending on why you are doing this, maybe a cp command would suffice.