jsoncpp write incrementally

Question

I have to log down what my application does into a json file. Is expected that the application goes on for weeks and so I want to write the json file incrementally.

For the moment I'm writing the json manually, but there is some log-reader application that is using Jsoncpp lib and should be nice to write the log down with Jsoncpp lib too.

But in the manual and in some examples I haven't found anything similar.. It is always something like:

Json::Value root;
// fill the json

ofstream mFile;
mFile.open(filename.c_str(), ios::trunc);
mFile << json_string;
mFile.close();

That is not what I want because it unnecessary fills the memory. I want to do it incrementally.. Some advice?

The main issue is that the JSON data format specify a closing token '}'. So, if you want your log file to be a valid JSON block after each of your incremental write, you need to open the file, remove the closing token, append the data and add the closing token. I did not find anything about that in the jsoncpp source code. — Matthieu Rouget, May 22 '13 at 10:49
Check out http://stackoverflow.com/questions/6886283/how-i-can-i-lazily-read-multiple-json-objects-from-a-file-stream-in-python — ctn, May 22 '13 at 11:40
Make your log format sequential JSON objects rather than one big one. Easy. — Lightness Races in Orbit, Feb 14 '15 at 19:52

cdunn2001 · Answer 1 · 2015-02-15T00:36:09.190

5

I am a maintainer of jsoncpp. Unfortunately, it does not write incrementally. It does write into a stream without using extra memory, but that doesn't help you.

edited Feb 15 '15 at 00:36

answered Feb 14 '15 at 19:47

cdunn2001

17,657
8
55
45

—1: JsonCpp is excellent, but this advertisement doesn't in any way answer the question. – Lightness Races in Orbit Feb 14 '15 at 19:53
@LightnessRacesinOrbit, Better now? – cdunn2001 Feb 15 '15 at 00:37
Well sort of but now it's just an answer admitting that it's not an answer :P – Lightness Races in Orbit Feb 15 '15 at 00:39
@LightnessRacesinOrbit, But that's the answer! Of course, one could write `[`, then write each value to the stream, usually add `,`, and finally write `]`. But I think the OP wants a library to do that automatically. At least *jsoncpp* actually streams the value instead of writing into a large buffer. – cdunn2001 Feb 15 '15 at 00:53
Oh, yes, you're right it is the answer. I'd forgotten that this question was specifically about JsonCpp since our original interaction. – Lightness Races in Orbit Feb 15 '15 at 00:55

score 3 · Accepted Answer · edited May 23 '17 at 12:22

If you can switch to plain JSON to JSON lines, as described in How I can I lazily read multiple JSON objects from a file/stream in Python? (thanks ctn for the link), you can do something like that :

const char* myfile = "foo.json";

// Write, in append mode, opening and closing the file at each write
{   
    Json::FastWriter l_writer;
    for (int i=0; i<100; i++)
    {
        std::ofstream l_ofile(myfile, std::ios_base::out | std::ios_base::app);

        Json::Value l_val;
        l_val["somevalue"] = i;
        l_ofile << l_writer.write(l_val);

        l_ofile.close();
    }       
}

// Read the JSON lines
{
    std::ifstream l_ifile(myfile);
    Json::Reader l_reader;
    Json::Value l_value;
    std::string l_line;
    while (std::getline(l_ifile, l_line))
        if (l_reader.parse(l_line, l_value))
            std::cout << l_value << std::endl;  
}

In this case, you do not have a single JSON in the file anymore... but it works. Hope this helps.

jsoncpp write incrementally

2 Answers2