docker-compose to create replication in mongoDB

Question

I am setting up a docker container to initialize the Mongo DB replica set using docker compose. I am not able to get this done. I have created docker-compose to invoke three mongo instance but i am getting parser issue. Below is my Docker-Compose file.

    version: '3'
    services:
     mongo-vauto-1:    
        image: "mongo-start"
        build: ./mongo-vAUTO-1
        ports:
          - "30000:27017"
        volumes:
          - ./mongo-vAUTO-1/data:/data/db
        depends_on:
          - "mongo-vauto-2"
          - "mongo-vauto-3"
    
     mongo-vauto-2:
        image: "mongo"
        command: --replSet vAUTO --smallfiles --oplogSize 128
        ports:
          - "40000:27017"
        volumes:
          - ./mongo-vAUTO-2/data:/data/db
    
     mongo-vauto-3:
        image: "mongo"
        command: --replSet vAUTO --smallfiles --oplogSize 128
        ports:
          - "50000:27017"
        volumes:
          - ./mongo-vAUTO-3/data:/data/db
    
     mongo-setup-vauto:
        image: "mongo-setup-vauto"
        build: ./setup
        depends_on:
          - "mongo-vauto-1"

which invokes Dockerfile which is in ./setup folder. Below is my set up Dockerfile

    FROM mongo

    # Create app directory
    WORKDIR vauto-docker/src/config
    
    # Install app dependencies
    COPY replicaSet.js .
    COPY setup.sh .
    
    CMD ["./setup.sh"]

which invokes replicaSet.js and setup.sh. The replicaSet.js and setup.sh files are provided below

    # replicaSet.js
    rsconf = {
        _id : "vAUTO",
        members: [
            {_id : 0, host : "mongo-vauto-1:27017"},
            {_id : 1, host : "mongo-vauto-2:27017"},
            {_id : 2, host : "mongo-vauto-3:27017"}
        ]
    }
    rs.initiate(rsconf);
    rs.conf();


    #  setup.sh 
    echo *********************************
    echo Starting the replica set
    echo *********************************
    
    sleep 10 | echo waiting for the replica set to be intialized
    mongo mongodb://mongo-vauto-1:27017 replicaSet.js

Dockerfile for first replica set is given below

    FROM mongo
    WORKDIR vauto-docker/src/config
    copy mongo.conf .
    EXPOSE 27017
    CMD ["--config","./mongo.conf"]

and my mongo.conf file has the following code

    replication:
      oplogSizeMB: 1024
      replSetName: vAUTO

I am getting the following error,

    mongo-vauto-1_1      | parse error: Invalid numeric literal at line 1, column 14
    mongo-vauto-1_1      | parse error: Invalid numeric literal at line 1, column 14
    mongo-vauto-1_1      | Error parsing YAML config file: yaml-cpp: error at line 3, column 1: illegal tab when looking for indentation

Use https://codebeautify.org/yaml-validator and validate your compose file, if not done. — Sameer K, Apr 30 '19 at 08:05
Possible duplicate of [replica Set mongo docker-compose](https://stackoverflow.com/questions/42190267/replica-set-mongo-docker-compose) — Akshay barahate, Apr 30 '19 at 09:23

score 21 · Accepted Answer · edited Sep 17 '21 at 11:02

Here is how I did this using a 4th container to initialize the replica set after 3 mongodb containers are running:

Docker-compose file

    version: '3'
    services:
    
      mongodb1:
        image: mongo:latest
        networks:
          - alphanetwork
        volumes:
          - data1:/data/db
          - config1:/data/configdb
        ports:
          - 30001:27017
        entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
    
      mongodb2:
        image: mongo:latest
        networks:
          - alphanetwork
        ports:
          - 30002:27017
        entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
    
      mongodb3:
        image: mongo:latest
        networks:
          - alphanetwork
        ports:
          - 30003:27017
        entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
    
      mongoclient:
        image: mongo
        networks:
          - alphanetwork
        depends_on:
          - mongodb1
          - mongodb2
          - mongodb3
        volumes:
          - ./deployment_scripts:/deployment_scripts
        entrypoint:
          - /deployment_scripts/initiate_replica.sh
    
    networks:
      alphanetwork:
    
    volumes:
      data1:
      config1:

initiate_replica.sh

    #!/bin/bash
    
    echo "Starting replica set initialize"
    until mongo --host mongodb1 --eval "print(\"waited for connection\")"
    do
        sleep 2
    done
    echo "Connection finished"
    echo "Creating replica set"
    mongo --host mongodb1 <<EOF
    rs.initiate(
      {
        _id : 'rs0',
        members: [
          { _id : 0, host : "mongodb1:27017" },
          { _id : 1, host : "mongodb2:27017" },
          { _id : 2, host : "mongodb3:27017" }
        ]
      }
    )
    EOF
    echo "replica set created"

And don't forget to chmod +x ./deployment_scripts/initiate_replica.sh to give docker the permission to execute it.

this worked beautifully for my needs! i was needing to test transactions with mongo and this allowed me to spin up a replica set locally. thanks! — Jacob, Apr 14 '21 at 13:43

score 2 · Answer 2 · answered Oct 01 '21 at 09:35

Based on @ToDevAndBeyond solution, this is the working solution with access control (in this case the solution has only two instances):

Docker-compose file

version: '3.5'
services:
  mongodb1:
    image: myMongoImage
    command: [ "--replSet", "rs0", "--bind_ip_all", "--keyFile", "/data/custom_config/mongodb.key", "--port", "27018" ]
    network_mode: host
    ports:
      - 27018:27018
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: <username>
      MONGO_INITDB_ROOT_PASSWORD: <password>
  mongodb2:
    image: myMongoImage
    command: sh -c 'sleep 5 && /usr/local/bin/docker-entrypoint.sh --replSet rs0 --bind_ip_all --keyFile /data/custom_config/mongodb.key --port 27019'
    network_mode: host
    ports:
      - 27019:27019
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: <username>
      MONGO_INITDB_ROOT_PASSWORD: <password>
  mongoclient:
    image: mongo:4.0.23-xenial
    network_mode: host
    depends_on:
      - mongodb1
      - mongodb2
    volumes:
      - <path_to_script_folder>:/deployment_scripts
    entrypoint:
      - /deployment_scripts/initiate_replica.sh

initiate_replica.sh

#!/bin/bash

sleep 10

echo "Starting replica set initialize"
until mongo  --username <username> --password <password> --host localhost --port 27018 --eval "print(\"waited for connection\")"
do
    sleep 2
done
echo "Connection finished"
echo "Creating replica set"
mongo  --username <username> --password <password> --host localhost --port 27018 <<EOF
rs.initiate(
  {
    _id : 'rs0',
    members: [
      { _id : 0, host : "localhost:27018" },
      { _id : 1, host : "localhost:27019" }
    ]
  }
)
EOF
echo "replica set created"

Dockerfile (myMongoImage)

FROM mongo:4.0.23-xenial

RUN cd /
RUN mkdir /data/custom_config/
RUN openssl rand -base64 741 > /data/custom_config/mongodb.key
RUN chmod 600 /data/custom_config/mongodb.key
RUN chown mongodb:mongodb /data/custom_config/mongodb.key

Important: the mongodb server is started twice during startup; the first mongodb server is started to create the user and runs always on port 27017 and the other on 27018 or 27019; due to network_mode=host, the two docker containers have to start with a delay as otherwise both would try to start the server on port 27017 and one would silently fail.

As mentioned, don't forget to chmod +x ./deployment_scripts/initiate_replica.sh to give docker the permission to execute it.

It could be helpful also to know, that the owner and group of ./deployment_scripts/initiate_replica.sh is mongodb (as you can see in the Dockerfile)

I hope this could be useful for someone.

docker-compose to create replication in mongoDB

2 Answers2

Linked