0

I'm writing a logging program for a microcontroller with OS Linux. There is also a calculation function, in which those results shall stored on HDD and loaded when the logger is restarted.

My problem is, when I unplug the µC from current meanwhile the µC is overwritting some data, the overwritten data could be lost.

So how I may overwrite some data, but ensure whether the overwritten data or the written data is consistent if a unplug meanwhile the µC is overwritting happens?

Programming language is C++, so I would be in love if there is an boost library or even better a stl type.

JulianW
  • 897
  • 9
  • 23

1 Answers1

1

Use stream << flush; to flush the C++ output buffer to the OS, and use Linux fsync() to flush from the OS buffer to disk.

The latter requires a Unix file descriptor, so you'll need to use an implementation-dependent method to get the FD from the C++ stream. See Retrieving file descriptor from a std::fstream

For additional protection you need to use a fault-resistent filesystem with journaling. See https://www.ibm.com/developerworks/library/l-journaling-filesystems/index.html for an example.

Barmar
  • 741,623
  • 53
  • 500
  • 612
  • I know those commands and I think this isn't answering my question: What is happening if a voltage failure is happening meanwhile the OS is overwriting some data? I think the data is lost, but that's exactly what I don't want. I don't want simply append data, I want overwrite existing data, because the amount of data of this specific type is too big. I don't want implement my own voltage failur safe file overwriting class. – JulianW Nov 06 '18 at 17:32
  • I'm writing this type each 5ms, so the probability of unplugging the µC meanwhile it is overwriting some data, is very high. – JulianW Nov 06 '18 at 17:42
  • 1
    You need to use a fault-tolerant journaling filesystem. I've added a link to an article about this. – Barmar Nov 06 '18 at 17:45
  • @JulianH It sounds to me like you need to open a stream with buffering disabled and add an IO card to your computer which is equipped with a battery so that the IO operation can be completed when the system comes back up from the voltage fluctuation. Also, if you are worried about overwrites as you said - then you want to implement some type of transaction log. A simple example is where data/file/to-be-modified-portion-of-file is "backed up" before the write occurs and then the backup is deleted upon successful IO operation. – NTDLS Nov 06 '18 at 17:47
  • You should also research the techniques used by database management systems. This is getting much too broad for a simple SO answer. – Barmar Nov 06 '18 at 17:49
  • I can't believe that there is no software solution for this? Like Barmar said, how database management system are solving this problem? SQLite or mongodb should have the same problem. I'm free to use any library... A other idea is to adding a condensator to the board, so the program may shutdown normally. – JulianW Nov 06 '18 at 17:54
  • It should be resolved by filesystem with journal. If transaction is not fully completed, then there is rollback on next boot when fscking dirty fs. – DevilaN Nov 06 '18 at 18:13
  • How big those journals are? What is happening if I try to write multiple mega bytes? I think there are happening more than one OS flush before the whole data is written, so the consistency of the data can't be ensured. Or am I wrong? – JulianW Nov 08 '18 at 14:48
  • I don't know the detailed design, but my guess is it just uses whatever free space is available on the disk. If you run out, you'll give an error and nothing will be written. – Barmar Nov 08 '18 at 16:15