How to create a queue in RabbitMQ upon startup

Question

I am trying to start RMQ inside docker container, with precreated queue qwer.

Prior to this, I was using simple docker-compose.yml file:

rabbit:
    image: rabbitmq:management-alpine
    environment:
      RABBITMQ_DEFAULT_USER: guest
      RABBITMQ_DEFAULT_PASS: guest

And it worked fine, except that it has no queues pre-created at start. Now I've switched to custom image, with following Dockerfile:

FROM rabbitmq:management-alpine

ADD rabbitmq.conf /etc/rabbitmq/
ADD definitions.json /etc/rabbitmq/

RUN chown rabbitmq:rabbitmq /etc/rabbitmq/rabbitmq.conf /etc/rabbitmq/definitions.json

where rabbitmq.conf is v3.7+ sysctl-styled config, with line:

management.load_definitions = /etc/rabbitmq/definitions.json

and definitions.json contains attempt to create queue:

{
    "vhosts":[
        {"name":"/"}
    ],
    "queues":[
        {"name":"qwer","vhost":"/","durable":true,"auto_delete":false,"arguments":{}}
    ]
}

Now it started to refuse login:

Error on AMQP connection <0.660.0> (172.18.0.6:48916 -> 172.18.0.10:5672, state: starting):
PLAIN login refused: user 'guest' - invalid credentials

I thought that the task is somewhat simple, but configuration process of rabbit itself is most complex task, and documentation is somewhat unclear.

I was unable to figure out how should it work, even after 4 days of trials and googling..

Could you help me, how to write configuration file, in order to create a queue and preserve ability to connect and talk to it?

Answer 1

You can predefine queues and exchanges without creating own rabbit-mq docker image.

Your docker-compose should look like this:

rabbit:
container_name: rabbitmq-preload-conf
image: rabbitmq:3-management
volumes:
  - ./init/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
  - ./init/definitions.json:/etc/rabbitmq/definitions.json:ro
ports:
  - "5672:5672"
  - "15672:15672"

In this case rabbitmq.conf and definitions.json files should be in init folder in the same parent folder as docker-compose file

rabbitmq.conf file

management.load_definitions = /etc/rabbitmq/definitions.json

definitions.json file

{
"queues": [
    {
        "name": "externally_configured_queue",
        "vhost": "/",
        "durable": true,
        "auto_delete": false,
        "arguments": {
            "x-queue-type": "classic"
        }
    }
],
"exchanges": [
    {
        "name": "externally_configured_exchange",
        "vhost": "/",
        "type": "direct",
        "durable": true,
        "auto_delete": false,
        "internal": false,
        "arguments": {}
    }
],
"bindings": [
    {
        "source": "externally_configured_exchange",
        "vhost": "/",
        "destination": "externally_configured_queue",
        "destination_type": "queue",
        "routing_key": "externally_configured_queue",
        "arguments": {}
    }
  ]
 }

NOTE: After update in rabbit images, additional configuration may be required. If rabbit container failed to start with the configuration mentioned above (error message contains "Exception during startup: exit:{error,{no_such_vhost,<<"/">>}}") following configuration should be added into definitions file:

 "users": [
    {
      "name": "guest",
      "password_hash": "BMfxN8drrYcIqXZMr+pWTpDT0nMcOagMduLX0bjr4jwud/pN",
      "hashing_algorithm": "rabbit_password_hashing_sha256",
      "tags": [
        "administrator"
      ],
      "limits": {}
    }
  ],
  "vhosts": [
    {
      "name": "/"
    }
  ],
  "permissions": [
    {
      "user": "guest",
      "vhost": "/",
      "configure": ".*",
      "write": ".*",
      "read": ".*"
    }
  ]

Using this additional configuration, default user will be guest with password guest.

Apart from queues, exchanges and bindings, definitions.json file can contain additional configuration

Answer 2

You are almost there actually.

RabbitMQ has a rule that the "guest" user can only connect from localhost. Since you are running it on a docker, I'm assuming you are trying to access it from outside by exposing port "15672" by doing: docker run <rabbitmq-docker-img> -p 15672:15672

So to get around this, what you have to do is create a user with admin privileges.

Firstly, change this:

rabbit:
    image: rabbitmq:management-alpine
    environment:
      RABBITMQ_DEFAULT_USER: user
      RABBITMQ_DEFAULT_PASS: password

You can use ath, I used user/password as your user/password.

In your Dockerfile, you can add: EXPOSE 15672 If you don't want to expose each time you run.

Lastly, make amends to your definitions.json file as follows:

{
    "users": [
      {
        "name": "user",  
        "password_hash": "password",
        "hashing_algorithm": "rabbit_password_hashing_sha256",
        "tags": "administrator"
      }
    ],

    "vhosts":[
        {"name":"/"}
    ],
    "queues":[
        {"name":"qwer","vhost":"/","durable":true,"auto_delete":false,"arguments":{}}
    ]
}

Let me know how it goes!

Check out this link

Use this Dockerfile:

FROM rabbitmq

# Define environment variables.
ENV RABBITMQ_USER user
ENV RABBITMQ_PASSWORD password

ADD init.sh /init.sh
EXPOSE 15672

# Define default command
CMD ["/init.sh"]

And use this init.sh:

#!/bin/sh

# Create Rabbitmq user
( sleep 5 ; \
rabbitmqctl add_user $RABBITMQ_USER $RABBITMQ_PASSWORD 2>/dev/null ; \
rabbitmqctl set_user_tags $RABBITMQ_USER administrator ; \
rabbitmqctl set_permissions -p / $RABBITMQ_USER  ".*" ".*" ".*" ; \
echo "*** User '$RABBITMQ_USER' with password '$RABBITMQ_PASSWORD' completed. ***" ; \
echo "*** Log in the WebUI at port 15672 (example: http:/localhost:15672) ***") &

# $@ is used to pass arguments to the rabbitmq-server command.
# For example if you use it like this: docker run -d rabbitmq arg1 arg2,
# it will be as you run in the container rabbitmq-server arg1 arg2
rabbitmq-server $@

Answer 3

I found the accepted answers use of init.sh and committing the password unencrypted in the definitions.json file to be insufficient for my needs, and came up with the following solution instead:

Start with a vanilla RabbitMQ, configure its queues and users through the web UI, export the configuration and then copy it into a new docker image (or you could mount it into the base RabbitMQ image of your choice as a volume, if you prefer):

1. Start a container running the rabbitmq:3.9-management image and publish the admin interface port to the localhost:

docker run -p 8080:15672 rabbitmq:3.9-management

1. Visit http://localhost:8080 in a browser and login with username: guest, password: guest.

2. Configure and create the queue(s) in the RabbitMQ web UI under the Queues tab. Also create any additional users you might want.

3. Export the definitions under the Overview tab. Name the file definitions.json and place it in an empty directory conf:

$ tree .
.
└── conf
    └── definitions.json

4. Create a file, rabbitmq.conf, containing the single line:

load_definitions = /etc/rabbitmq/definitions.json

and place it in the conf directory:

$ tree .
.
└── conf
    ├── definitions.json
    └── rabbitmq.conf

5. Define a new docker image with the RabbitMQ definitions copied into it:

FROM rabbitmq:3.9-management
COPY conf/rabbitmq.conf /etc/rabbitmq/
COPY conf/definitions.json /etc/rabbitmq/

and a docker-compose.yml file to build the image and run the container:

version: "3.9"
services:

  my-rabbitmq:
    build: .
    ports:
    - 8082:15672

Directory structure:

$ tree .
.
├── Dockerfile
├── conf
│   ├── definitions.json
│   └── rabbitmq.conf
└── docker-compose.yml

1. Start the container:

$ docker-compose -f docker-compose.yml up --build

and visit the web UI to ensure the definitions have been loaded and the queue created.

---

If you don't want to configure your RabbitMQ through the UI, the definitions.json file is pretty self-explanatory, so it should be straight forward to edit it in order to add new queus, creating new users etc., though you need to see the answer to this question on how to create the sha256 hash.