1

I'm working on a new project (a game engine for self education) and trying to create a logging system. I want the logger to help with debugging as much as possible, so I plan on using it a lot to write to a log file. The only issue is that I'm worried doing file I/O will slow down the game loop which needs to operate within a time bound. What is the best way I can write to a file with minimal risk of slowing down the important section?

I have thought about using threads, but I'm worried that the overhead of context switched due to the process scheduler may be even more of an impediment to performance.

I have considered writing to a buffer and occasionally doing a large dump to the file, but I have read that this can potentially be even slower than regular file writing if the buffer becomes too big. Is it feasible to keep the whole buffer in memory and only write all the contents to the file at once at the end of the program?

I have read lightly about using a memory mapped file, but I've also read that it requires the boost library to be done effectively. I'd like to minimize the dependencies, so ideally I wouldn't use boost. I'm also not entirely sure that my concept of memory mapped files is correct. From what I understand, it behaves as if you are simply writing to memory, but eventually the memory contents will be written to the file. Is this conception correct?

Thanks for reading all of this :)

TL;DR - How can I implement a logging system that minimizes the performance decrease of my program?

Kejoko
  • 11
  • 1
  • 3
  • This question is too broad to answer. I can only give rough advice: Move your logging to a thread. Communicate through a thread-safe structure. Don't flush unless you need to. Just use regular buffered writes, nothing fancy. Above all else **never spend time optimizing code unless you have a measurable performance problem**. – tadman Jun 23 '20 at 02:20
  • This answer may help https://stackoverflow.com/questions/1201261/what-is-the-fastest-method-for-high-performance-sequential-file-i-o-in-c – KarlM Jun 23 '20 at 02:22
  • A "logger for debugging" and efficiency are often mutually exclusive - a logger needs to ensure data is correctly written to file, so that file can then be used to find problems - for example, if the program crashes. That reduces options for gaining efficiency, like buffering. You can do things for perceived (not actual) efficiency like having logging functions queue output in one thread, and another thread do the output. But that increases risk of lost output (can defeat purpose of logging) and a lot of work is needed to debug the logging system itself and its interactions with other code – Peter Jun 23 '20 at 03:17
  • @tadman Is it possible to implement a thread safe buffer that doesn't require the main thread to wait on the logging thread to signal? – Kejoko Jun 24 '20 at 03:03
  • The whole idea of using a thread-safe queue is there's no locks. There's other structures that require minimal locking or use other tools like atomic variables, to achieve their thread safety. – tadman Jun 25 '20 at 05:16

2 Answers2

0

If you decide to write everything to memory and at the end write the whole logs to the file, then if any application crash will wipe away all the debug data.

About the memory mapped file, you are write. But you have to consider when the in-memory pages will be written to the disk.

user1288043
  • 181
  • 1
  • 3
  • 10
0

You can use from Ipc methods and separate the logger process from main process and these two process communicate with each other via a queue. main process put the message in queue and logger process get the message and write to file.

Farhad Sarvari
  • 1,051
  • 7
  • 13
  • Wouldn't I need to protect the queue with a mutex, causing the main thread to occasionally wait for the logging thread to signal? – Kejoko Jun 24 '20 at 03:01
  • It's not necessary to protect the queue. You have two process work separately one for writing and another for reading from the queue and OS handles it. – Farhad Sarvari Jun 24 '20 at 04:46