boost program_options: Read required parameter from config file

Question

I want to use boost_program_options as follows:

• get name of an optional config file as a program option
• read mandatory options either from command line or the config file

The problem is: The variable containing the config file name is not populated until po::notify() is called, and that function also throws exceptions for any unfulfilled mandatory options. So if the mandatory options are not specified on the command line (rendering the config file moot), the config file is not read.

The inelegant solution is to not mark the options as mandatory in add_options(), and enforce them 'by hand' afterwards. Is there a solution to this within the boost_program_options library?

MWE

bpo-mwe.conf:

db-hostname = foo
db-username = arthurdent
db-password = forty-two

Code:

#include <stdexcept>
#include <iostream>
#include <fstream>
#include <filesystem>
#include <string>
#include <boost/program_options.hpp>

// enable/disable required() below
#ifndef WITH_REQUIRED
#define WITH_REQUIRED
#endif

namespace po = boost::program_options;
namespace fs = std::filesystem;

int main(int argc, char *argv[])
{

    std::string config_file;

    po::options_description generic("Generic options");
    generic.add_options()
    ("config,c", po::value<std::string>(&config_file)->default_value("bpo-mwe.conf"), "configuration file")
    ;

    // Declare a group of options that will be
    // allowed both on command line and in
    // config file
    po::options_description main_options("Main options");
    main_options.add_options()
    #ifdef WITH_REQUIRED
    ("db-hostname", po::value<std::string>()->required(), "database service name")
    ("db-username", po::value<std::string>()->required(), "database user name")
    ("db-password", po::value<std::string>()->required(), "database user password")
        #else
    ("db-hostname", po::value<std::string>(), "database service name")
    ("db-username", po::value<std::string>(), "database user name")
    ("db-password", po::value<std::string>(), "database user password")
    #endif
    ;

    // set options allowed on command line
    po::options_description cmdline_options;
    cmdline_options.add(generic).add(main_options);

    // set options allowed in config file
    po::options_description config_file_options;
    config_file_options.add(main_options);

    // set options shown by --help
    po::options_description visible("Allowed options");
    visible.add(generic).add(main_options);

    po::variables_map variable_map;

    // store command line options
    // Why not po::store?
    //po::store(po::parse_command_line(argc, argv, desc), vm);
    store(po::command_line_parser(argc, argv).options(cmdline_options).run(), variable_map);

    notify(variable_map); // <- here is the problem point

    // Problem: config_file is not set until notify() is called, and notify() throws exception for unfulfilled required variables

    std::ifstream ifs(config_file.c_str());
    if (!ifs)
    {
        std::cout << "can not open configuration file: " << config_file << "\n";
    }
    else
    {
        store(parse_config_file(ifs, config_file_options), variable_map);
        notify(variable_map);
    }

    std::cout << config_file << " was the config file\n";
    return 0;
}

Answer 1

I'd simply not use the notifying value-semantic to put the value in config_file. Instead, use it directly from the map:

auto config_file = variable_map.at("config").as<std::string>();

Now you can do the notify at the end, as intended:

Live On Coliru

#include <boost/program_options.hpp>
#include <fstream>
#include <iomanip>
#include <iostream>

namespace po = boost::program_options;

int main(int argc, char *argv[])
{
    po::options_description generic("Generic options");
    generic.add_options()
        ("config,c", po::value<std::string>()->default_value("bpo-mwe.conf"), "configuration file")
    ;

    // Declare a group of options that will be allowed both on command line and
    // in config file
    struct {
        std::string host, user, pass;
    } dbconf;

    po::options_description main_options("Main options");
    main_options.add_options()
        ("db-hostname", po::value<std::string>(&dbconf.host)->required(), "database service name")
        ("db-username", po::value<std::string>(&dbconf.user)->required(), "database user name")
        ("db-password", po::value<std::string>(&dbconf.pass)->required(), "database user password")
    ;

    // set options allowed on command line
    po::options_description cmdline_options;
    cmdline_options.add(generic).add(main_options);

    // set options allowed in config file
    po::options_description config_file_options;
    config_file_options.add(main_options);

    // set options shown by --help
    po::options_description visible("Allowed options");
    visible.add(generic).add(main_options);

    po::variables_map variable_map;

    //po::store(po::parse_command_line(argc, argv, desc), vm);
    store(po::command_line_parser(argc, argv).options(cmdline_options).run(),
          variable_map);

    auto config_file = variable_map.at("config").as<std::string>();

    std::ifstream ifs(config_file.c_str());
    if (!ifs) {
        std::cout << "can not open configuration file: " << config_file << "\n";
    } else {
        store(parse_config_file(ifs, config_file_options), variable_map);
        notify(variable_map);
    }

    notify(variable_map);
    std::cout << config_file << " was the config file\n";

    std::cout << "dbconf: " << std::quoted(dbconf.host) << ", " 
        << std::quoted(dbconf.user)  << ", "
        << std::quoted(dbconf.pass)  << "\n"; // TODO REMOVE FOR PRODUCTION :)
}

Prints eg.

$ ./sotest
bpo-mwe.conf was the config file
dbconf: "foo", "arthurdent", "forty-two"

$ ./sotest -c other.conf 
other.conf was the config file
dbconf: "sbb", "neguheqrag", "sbegl-gjb"

$ ./sotest -c other.conf --db-user PICKME
other.conf was the config file
dbconf: "sbb", "PICKME", "sbegl-gjb"

Where as you might have guessed other.conf is derived from bpo-mwe.conf by ROT13.

Answer 2

Differentiate between configuration file and command-line arguments, don't parse both into the same map.

Instead first parse the command-line arguments separately, get the configuration file name (if there is any) and then load the file and parse it into a second map.

---

If some configuration-file values can be provided on the command line as well, then I personally do two passes over the command-line arguments, making it a three-step process:

1. Parse command-line arguments, ignore all but the config option
2. Read and parse the configuration file
3. And do a second pass over the command-line arguments, ignoring the config option