How do I make a non concurrent print to file system in C++?

Question

I am programming in C++ with the intention to provide some client/server communication between Unreal Engine 4 and my server.

I am in need of a logging system but the current ones are flooded by system messages. So I made a Logger class with a ofstream object which I do file << "Write message." << endl.

Problem is that each object makes another instance of the ofstream and several longer writes to the file get cut off by newer writes.

I am looking for a way to queue writing to a file, this system/function/stream being easy to include and call.

Bonus points: the ofstream seems to complain whenever I try to write std::string and Fstring :|

Is there any reason you can't just share a static instance of your `ofstream`? Or how about a simple FIFO buffer? — Mario, Aug 08 '14 at 14:02
Try using an existing logging framework, which already handles such things. — Some programmer dude, Aug 08 '14 at 14:03
The point of logging systems is that they can handle everything being thrown at the log, and filter out stuff you don't want to log. There should be a way to turn on/off all the system messages, or just enable the messages relevant to what you are currently debugging. You may need to figure out their configuration file. — Kenny Ostrom, Aug 08 '14 at 14:14
@DOOM I have not. I will try when I get home. Seems more logical, but does this delay the next append or interpolates them? — Discipol, Aug 08 '14 at 14:29

score 1 · Answer 1 · answered Aug 08 '14 at 14:02

1

log asynchronously using i.e. g2log or using a non-blocking socket wrapper, such as zeromq

answered Aug 08 '14 at 14:02

Dmitry Ledentsov

3,620
18
28

score 0 · Answer 2 · edited May 23 '17 at 12:20

0

ofstream can't be used across multiple threads. It needs to be synchronized using mutex or similar objects. Check the below thread for details:ofstream shared by mutiple threads - crashes after awhile

edited May 23 '17 at 12:20

Community

1
1

answered Aug 08 '14 at 14:04

dvasanth

1,337
1
9
10

score 0 · Answer 3 · answered Aug 08 '14 at 14:35

I wrote a quick example of how you can implement something like that. Please keep in mind that this may not be a final solution and still requires additional error checking and so on ...

#include <concurrent_queue.h>
#include <string>
#include <thread>
#include <fstream>
#include <future>

class Message
{
public:
    Message() : text_(), sender_(), quit_(true) 
    {}
    Message(std::string text, std::thread::id sender)
        : text_(std::move(text)), sender_(sender), quit_(false)
    {}

    bool isQuit() const { return quit_; }
    std::string getText() const { return text_; }
    std::thread::id getSender() const { return sender_; }

private:
    bool quit_;
    std::string text_;
    std::thread::id sender_;
};


class Log
{
public:
    Log(const std::string& fileName)
        : workerThread_(&Log::threadFn, this, fileName)
    {}
    ~Log()
    {
        queue_.push(Message()); // push quit message
        workerThread_.join();
    }

    void write(std::string text)
    {
        queue_.push(Message(std::move(text), std::this_thread::get_id()));
    }

private:
    static void threadFn(Log* log, std::string fileName)
    {
        std::ofstream out;
        out.open(fileName, std::ios::out);
        assert(out.is_open());
        // Todo: ... some error checking here

        Message msg;
        while(true)
        {
            if(log->queue_.try_pop(msg))
            {
                if(msg.isQuit()) 
                    break;
                out << msg.getText() << std::endl;
            }
            else
            {
                std::this_thread::yield();
            }
        }
    }

    concurrency::concurrent_queue<Message> queue_;
    std::thread workerThread_;
};


int main(int argc, char* argv[])
{
    Log log("test.txt");
    Log* pLog = &log;

    auto fun = [pLog]()
    {
        for(int i = 0; i < 100; ++i)
            pLog->write(std::to_string(i));
    };

    // start some test threads
    auto f0 = std::async(fun);
    auto f1 = std::async(fun);
    auto f2 = std::async(fun);
    auto f3 = std::async(fun);

    // wait for all
    f0.get();
    f1.get();
    f2.get();
    f3.get();

    return 0;
}

The main idea is to use one Log class that has a thread safe write() method that may be called from multiple threads simultaneously. The Log class uses a worker thread to put all the file access to another thread. It uses a threadsafe (possibly lock-free) data structure to transfer all messages from the sending thread to the worker thread (I used concurrent_queue here - but there are others as well). Using a small Message wrapper it is very simple to tell the worker thread to shut down. Afterwards join it and everything is fine. You have to make sure that the Log is not destroyed as long as any thread that may possibly write to it is still running.

How do I make a non concurrent print to file system in C++?

3 Answers3