What is the best way to change application configurations in a CI environment

Question

I am currently doing a POC on Jenkins pipeline to figure out how to configure my product in a CI environment. The requirements of the pipeline are:

1. Checkout code from SVN
2. Compile the program
3. Deploy to a predefined location on the server
4. Change DB configurations (& maybe even other configs not identified yet) to point to the appropriate DB
5. Execute the program
6. Execute QA process to validate the output

I am currently having difficulty in achieving Point 4 above. All DB-related configurations reside in a database.xml file per program & a program can connect to 1 or more DBs.

Given that developers are free to check-in any DB configurations, I would still like my CI environment to point to a predefined DB to test against. I am unsure on how to dynamically change these configuration files to achieve this.

Please let me know if there are standard methods that others are also using to achieve the same.

TIA

Answer 1

Some approaches:

Properties using Advanced Platforms

Use some web platform like :

• zookeeper
  • http://www.therore.net/java/2015/05/03/distributed-configuration-with-zookeeper-curator-and-spring-cloud-config.html
• Spring Cloud
  • https://www.baeldung.com/spring-cloud-configuration
  • This is a java spring framework functionality in wich you can create properties file with configurations and configure your applications to read them.
• magi-properties-management
  • This is a java web system in which you can create environments and any key:value in each one. You just need configure your application in any language to read this values.
• cyber-properties-management
  • This is a nodejs application that allows you to store properties files (.properties .yml or .json) and then just consume them as rest endpoint from your applications.

With this approaches , when a change of configurations is required, you just need update the value in the system and restart your application. It is even possible a hot reload in java applications.

Properties from Environment variables

You can export your key:value properties as environment vars before starting the application :

export DATABASE_HOST=10.100.200.300
export LOG_DIR_LOCATION=/logs

And read it after after the application has started:

Java >> System.getEnv("DATABASE_HOST"); 
node.js >> process.evn.LOG_DIR_LOCATION
php >> getenv('DATABASE_HOST')

Properties from SCM

• Create some svn repositoty called development-configurations
• Upload your database.xml with development values
• In your application, put a database.xml with dummy values : localhost, etc
• Create a jenkins job and put the environment as an argument.
• In the same job download svn source code of your application.
• download svn repository called $environment-configurations. $environment will be your argument
• replace the database.xml inside of your application with database.xml of $environment-configurations repository.
• Just create another repositories for testing, uat and production. Job must be receive environment as an argument to choose the right database.xml

Properties from Database

Modify your applications to read configurations from some database instead of xml file

Properties from File System

Modify your application to read an external database.xml instead of the database.xml inside of your source code. With this approach you just need put the database.xml in some path of your server and delete it from your application source code.

Note

You can use these approaches not only for backend apps. You can use them for frontends applications:

Devops Variable Substitution for Frontend js applications