how to use multiple configuration files for RabbitMQ / Erlang

Question

I am trying to setup a Spring based Java application which uses a locally installed RabbitMQ server for delivering messages between nodes. As some of you already know, the rabbitmq.config file can be used to configure various parameters and is loaded by the underlying Erlang node which the Rabbit server runs on.

My problem is that I have a requirement that some of the configuration needs to be static and some needs to be dynamic, specifically, I need to be able to reconfigure the shovels running on the Rabbit server from time to time as a result of user interaction (i.e. I need to modify the configuration file programmatic-ally and reboot the rabbit server for it to take affect), but, I don't want to rewrite the static configuration every time (especially because I don't want the java code to read it).

I thought I had a solution from reading the Erlang configuration file manual (http://www.erlang.org/doc/man/config.html) which explains how to use one configuration file that points to another such that the configuration of both files will be merged by Erlang. Unfortunately, it doesn't seem to work at all and I could not find any reference to this problem online.

I am testing this on Windows 7 x64 OS using RabbitMQ 3.1.3 and Erlang 5.10/OTP R16.

1st config file:

[
{'rabbit', [
    {'tcp_listeners', [
        5672
    ]},
    {'default_vhost', <<"/">>}
]}, "C:\\Users\\itay\\Desktop\\RabbitMQ\\rabbitmq2.config"
].

2nd config file:

[
{'rabbit', [
    {'default_user', <<"guest">>},
    {'default_pass', <<"guest">>}
]}
].

I tried to use single backslash or bit-string for the path as well but it didn't seem to matter.

The output from running the server in cmd is:

{"could not start kernel pid",application_controller,"invalid config data: invalid application     name:  \"C:\\Users\\itay\\Desktop\\RabbitMQ\\rabbitmq2.config\""}

Crash dump was written to: erl_crash.dump
could not start kernel pid (application_controller) (invalid config data: invalid application name:  "C:\\Users\\itay\\Desktop\\RabbitMQ\\rabbitmq2.config")

Any kind of solution or assistance will be appreciated,

Thanks.

score 2 · Accepted Answer · edited Jun 20 '20 at 09:12

The config documentation you are linking to refers to sys.config file in embedded mode.

When starting Erlang in embedded mode, it is assumed that exactly one system configuration file is used, named sys.config. This file should be located in $ROOT/releases/Vsn, where $ROOT is the Erlang/OTP root installation directory and Vsn is the release version.

I doubt you are running RabbitMQ in embedded mode and judging from question, you are not editing sys.config file. You are probably editing RabbitMQ's default config file and using the default scripts to start the server (and therefore running erlang in interactive mode).

Instead, what you actually want is to pass specific configuration values to an application. The solution is on the same page you are linking, above:

A configuration file contains values for configuration parameters for the applications in the system. The erl command line argument -config Name tells the system to use data in the system configuration file Name.config.

Configuration parameter values in the configuration file will override the values in the application resource files (see app(4)). The values in the configuration file can be overridden by command line flags (see erl(1)).

So you can simply pass to erl on the command line either:

an additional -config parameter pointing to the second file

-config second_file
values with the -App Par Val syntax (both Par and Val are interpreted as terms, add quotes)

-rabbit default_user '<<"guest">>' -rabbit default_pass '<<"guest">>'

I guess you'll have to use RABBITMQ_SERVER_START_ARGS environment variable, or edit the rabbitmq-server.bat script or whatever your Java code uses to start RabbitMQ.

OK so as I understand when Erlang runs in **embedded** mode it always looks for sys.config and ignores the `-config` option. Therefore, it was made possible to have the sys.config file point to another file which will be merged with it. In **interactive** mode, the `-config` option is used and no merging is supported. Is this a correct description? — Itay, Aug 08 '13 at 15:12
I have tried to add another `-config` flag with a second config file but the node will not start - I get this output: `Conflicting -start_erl and -config options` Seems like only one config file is supported... This is bad for me since it is not practical to configure the shovels using only the `-App Par Val` arguments. — Itay, Aug 08 '13 at 15:13
[start_erl](http://www.erlang.org/doc/man/start_erl.html) is a Windows-specific tool to start Erlang in **embedded** mode. The error message implies you are passing `-config` with your second config file in a command line where there was no `-config` option at all, i.e. probably not at the proper place. Did you modify the script or did you use `RABBITMQ_SERVER_START_ARGS`? — Paul Guyot, Aug 09 '13 at 07:00
I used the start args variable. I can also see that rabbit uses -config to set the default rabbitmq.config file in the command line. However, your other suggestion might work well for me since it is possible to use the `-args_file` option and load the shovels from a separate file containing a single `-rabbit 'shovels' ` argument. I can even use the backslash character for adding line breaks to make this long expression humanly readable. — Itay, Aug 09 '13 at 10:16

how to use multiple configuration files for RabbitMQ / Erlang

1 Answers1