What is the best way to manage configuration data

Question

I am working on a product suite which has 4 products. Right now, all of the configuration data is either in the XML or properties files.This approach is not maintainable as we have to manage different configuration file for different environment(for e.g. Production, Development etc).

So, what is the best way to handle configuration data?

Also, can we modularize this into a separate module? So that all products can use this module. We don't want to use the property files. I am looking for a solution in which we can move all of the configuration specific code as a new configuration module and persist all the configuration data in database.

Answer 1

Using commons-configuration you have a unified API for accessing the properties, no matter how they are represented - .properties, xml, JNDI, etc. For example:

config.properties:

jdbcHost=192.168.12.35
jdbcUsername=dbuser
jdbcPassword=pass

config.xml:

<config>
   <jdbcHost>192.168.12.35</jdbcHost>
   <jdbcUsername>dbuser</jdbcUsername>
   <jdbcPassword>pass</jdbcPassword>
</config>

in both cases they will be accessible with something like:

String host = config.getString("jdbcHost");

Answer 2

You're almost there... I would keep your same approach and pull in the correct configuration file for the instance of the application that is running by a method similar to either of the following:

1. Name all of your configuration files differently and have your application pull them in by some unique criteria (username, hostname, etc.):

   • production.properties
   • developer1.properties
   • developer2.properties

2. Keep them outside the codebase in a location based on an environment variable that the application assumes exists:

   • YOURAPP_CONFIG_DIR/server_config.xml
   • YOURAPP_CONFIG_DIR/database_config.properties

I've even used a combination of these approaches on the same project (#1 for build process configurations and #2 for runtime configurations).

Answer 3

If your applications work with a database, you can create a "configuration" table as follows:

create table configuration (mode char(3), key varchar(255), value varchar(1023));

You would initialize it using an init script, say init.sql with contents along the lines of:

insert into configuration values ('pro', 'param1', 'value1'); -- production
insert into configuration values ('dev', 'param1', 'value1'); -- development
insert into configuration values ('tst', 'param1', 'value1'); -- testing
...

The benefits of this approach are as follows:

• you version the script together with your code
• you can easily extend it to include per-user or per-group settings by adding a user/group id
• you can change the settings at runtime if you need to do so
• you get to use the same stack (JPA + DAO, Cayenne...) you normally use to handle core application data to handle configuration data

Answer 4

For all of our environments, configuration data lives on the target machines in the form of properties files. We use PropertyPlaceholderconfigurer from SpringFramework to bind these properties to our apps to keep things portable accross environments.

For example, as long as I know that /etc/myapp/database.properties will be present on whatever machine my app will be running on, then in my spring configuration, I just need something like so:

    <bean id="myPropertyConfigurer"
    class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="locations">
        <list>
            <value>/etc/myapp/database.properties</value>
        </list>
    </property>
</bean>
<bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource">
    <property name="driverClassName" value="com.mysql.jdbc.Driver" />
    <property name="url"
        value="jdbc:mysql://${db.host}:3306/${db.name}" />
    <property name="username" value="${db.user}" />
    <property name="password" value="${db.pass}" />     
</bean>

There are a bunch of options for that Spring class about where properties files can live. You can even make them substitutions and pass them in as environment variables:

    <bean id="myPropertyConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
    <property name="searchSystemEnvironment" value="true" />
    <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
    <property name="locations">
        <list>
            <value>${database.configuration.file.url}</value>
        </list>
    </property>
</bean>

And in bash_profile (or whatever): export JAVA_OPTS="-Ddatabase.configuration.file.url=file:///etc/myapp/database.properties"

Or just the same -D option passed in when you call "java" depending on what you are doing.

FWIW, we maintain our properties files separately as RPMs.

Answer 5

There are plenty of different strategies. All of them are good and depends on what suit you best.

1. Build a single artifact and deploy configs to a separate location. The artifact could have placeholder variables and, on deployment, the config could be read in. Have a look at Springs property placeholder. It works fantastically for webapps that use Spring and doesn't involve getting ops involved.
2. Have an externalised property config that lives outside of the webapp. Keep the location constant and always read from the property config. Update the config at any stage and a restart will be up the new values.
3. If you are modifying the environment (i.e. application server being used or user/group permissions) look at using the above methods with puppet or chef. Also have a look at managing your config files with these tools.

Answer 6

Environment variables are just about the easiest way to go. Set them as you would any other time, access them w/ System.getenv("...")

Answer 7

Config is a configuration file management tool. You can create configuration that is common on all environments, or you can create an environment specific configuration. You can keep using your XML and properties files, and let Config maintain the differences in environment. You can think of Config as your centralized database and it can output the configuration file in the format that you want. Whenever you want your configuration file, just deploy (push or pull) it from Config to your desired location. Note that I'm part of the Config team.