Multithreaded Logging Help

Question

I am working on a multithreaded logging for my engine. But I ma having troubles making it MT. The problem is, even if I am deep copying my logger object into a local variable I am going to have problems with the files. Because two threads are going to write to the same file at the same time which will build a mess. Here is my logger class:

class Logger {
public:
    typedef std::vector<LogListener *> ListenerList;

private:
    ListenerList listeners;
            boost::mutex mutex;

public:
    Logger();
    ~Logger();

    Logger * write(const String &line);

};

Logger * Logger::write(const String &text) {

    if(listeners.empty()) return this;
    boost::unique_lock<boost::mutex> lock(mutex);
    for(ListenerList::iterator i = listeners.begin(); i != listeners.end(); ++i) {
        (*i)->write(text);
    }

    return this;

}

class FileLogListener : public LogListener {
    std::ofstream stream;
public:
    FileLogListener(const String &str) : stream(str.c_str(), std::ios::out | std::ios::app) { }
    void write(const String &text) {
        stream << text << std::endl;
    }

};

Now suppose (avoiding mutexes):

//Thread 1

void func1() {
    Logger * log = new Logger;
    log->addListener("file.txt");
    log->write("Thread 1 Test");
}

//Thread 2

void func2() {
    Logger * log = new Logger;
    log->addListener("file.txt");
    log->write("Thread 2 test");
}

int main() {
  boost::thread t1(&func1);
  boost::thread t2(&func2);

  t1.join();
  t2.join();
  return 0;
}

"file.txt" becomes a mess.

EDIT: Currently, I am reading and watching lectures on Multithreading to understand it better.

EDIT: The above logger works.

Thanks in advance,
Gasim Gasimzada

Answer 1

Either don't write to the same file, or serialise the writes (e.g. by using a lock).

Answer 2

once I've written this article about minimalist logger. it's a single header file simple logger that can be used in MT environment. it satisfied my logging needs in lots of projects

If you want to use your logger, you can just check the approach

in short: it just locks access to the file by boost::mutex

Answer 3

you should give log4cxx a chance.

i just know the java und the c# version of this logger framework, but they are great.

and you don't have to concern about multithreading and can concentrate on your actual work.

you also easily can change the log destination from file to sql or even udp in just changing the configuration file.

http://logging.apache.org/log4cxx/index.html

hth

Answer 4

Create a singleton Logger object per file and have that hold a mutex in its write method.

Answer 5

In Ubuntu Linux there's probably a syslog daemon running. Utilize it, don't reinvent the wheel.

Answer 6

I think the difficulty you are having is that you have multiple file liseners that all point to the same file. This means that you don't have a mutex that catches all attempted writes to the file. One way round this is to add a static mutex to your lisener:

class FileLogListener : public LogListener {
    std::ofstream stream;
    static  boost::mutex m_mutex; //same mutex for all writes to (any) file
public:
    FileLogListener(const String &str) : stream(str.c_str(), std::ios::out | std::ios::app) { }
    void write(const String &text) {
        boost::mutex::scoped_lock lock(m_mutex);//lock the mutex
        stream << text << std::endl;
    }

};

 //somewhere in  you cpp file, initialize the mutex
 boost::mutex FileLogListener::m_mutex;

I also think the logger you have set up is a overly complicated (maybe you have a reason beyond the scope of this question). A problem you might have in the future is that you might find it difficult to turn logging off when you want to optimize you code.

Just in case you find it useful, I thought I would post an alternative,

#include <iostream>
#include <fstream>
#include <boost/thread.hpp>
template<class T>
struct threadsafe
{
  static boost::mutex m_mutex;
  static void print(const std::string& msg) 
  {
    boost::mutex::scoped_lock lock(m_mutex);
    T::print(msg);
  }
};


struct file_print
{
  static std::ofstream stream;
  static void print(const std::string& msg) 
  {
    stream<<msg;
  }
};
struct console_print
{
  static void print(const std::string& msg) 
  {
    std::cout<<msg<<std::endl;
  }
};


struct dont_log
{
  static void print(const std::string& msg) 
  {}
};

//somewhere in the library c++ file
template<class T>
boost::mutex threadsafe<T>::m_mutex;

You can then use the logger like so:

template<class logger = dont_log>
struct my_class
{

  void run()
  {
    logger::print("msg\n");
  }

};

std::ofstream file_print::stream("test.txt");

int
main  (int ac, char **av)
{

  my_class< threadsafe<file_print> > c;
  //or log to the console
  //my_class< threadsafe<console_print> > c;
  //or don't log at all (by default) 
  //my_class<> c;
  c.run();
}

The advantage with this logger is that if you (or your users) decide they no longer want to log, they just switch the template parameter to dont_log. Since the print function is static, the compiler should remove all references to the empty function, and optimise all logging away.