3

I'm trying to write out raw video frames (12.4MB each) in realtime (30fps) to a CFast 2.0(ext4) card which is connected via a sata3(6Gb/sec) interface.

The card is rated at 430MB/sec and when I benchmark the drive with consecutive 100MB file writes, it happily reaches ~420MB/sec.

The problem is when I'm writing smaller files, ~12MB, the throughput drops to about 350MB/sec which becomes too slow for my purposes.

The file writing routine is relatively simple fopen based(pseudocode):

foreachframe()
{
    file = fopen(frame_filename)
    fwrite(file, img_header)
    fwrite(file, img_data)
    fclose(file)
}

I've tried both single threaded and multi threaded, but there is not much difference. I'm guessing there is some significant overhead for creating a new file and closing it. Currently the filesystem is ext4, although I'd like to get it working with exFat ultimately.

Is there a way to interact low level with the filesystem that would allow creating and filling large numbers files with a much lower overhead? Alternatively are there optimization tricks for batch-saving a large number of files to disk?

Mark Setchell
  • 191,897
  • 31
  • 273
  • 432
Russell Newman
  • 71
  • 1
  • 3
    Is it possible you could write a single file and if required post process it to split it up later? – Richard Critten Feb 07 '18 at 14:23
  • 2
    It'd be interesting to see if open()/write()/close() is faster than the buffered versions. Should be easy to test. – Bjorn A. Feb 07 '18 at 14:24
  • 1
    Make sure that you mount with `nobarrier,noatime,data=writeback`. If that's still too slow you could look into writing to the raw disk with no filesystem. Just space the frames on 12.4mb boundaries. – stark Feb 07 '18 at 14:25
  • Use two cards ;-) Or maybe split it into 2 partitions with 2 filesystems and see if that helps. – Mark Setchell Feb 07 '18 at 14:29
  • The issue may not be the data transfer rate, but during file creation. If the drive is clean, the blocks for allocation should be sequential. However, there may be times when the disk controller needs to find a suitable block to write to. If the drive has a mirror image of the directory structure in memory, the speed is good; however, if the drive needs to read any directory header information, it will take time to reposition and read again. – Thomas Matthews Feb 07 '18 at 14:55
  • @ThomasMatthews `If the drive has a mirror image of the directory structure in memory` do you mean something like `mmap()`? – piwi Feb 07 '18 at 14:56
  • 1
    Make sure you write data in few large chunks rather than many small ones. – Jesper Juhl Feb 07 '18 at 14:58
  • Maybe you could put the headers on at the end - after your high speed acquisition has finished. – Mark Setchell Feb 07 '18 at 15:24
  • As a variation of Richard Critten's suggestion, perhaps allow each file to contain N video frames rather than just 1, and experiment with different values of N until you find the right tradeoff between performance and flexibility. – Jeremy Friesner Feb 07 '18 at 15:30
  • Are you sure that problem dome from disk throughout? What is your CPU charge when you write multiple files? If you are sure it does not come from CPU usage, you can try to preallocate inode and data space using `fallocate(2)` (but I don't it can justify 20% of performance loss) – Jérôme Pouiller Feb 07 '18 at 15:47
  • Have you tried `ext2` or disabled `ext4` journalling? – Mark Setchell Feb 07 '18 at 16:00
  • @RichardCritten I might consider that as a last resort, although the invidual files is the target for now as I'm trying to make a Adobe Cinema DNG sequence. – Russell Newman Feb 07 '18 at 16:12
  • I'll try a combination of disabling journaling and barriers along with the fallocate pre-allocation – Russell Newman Feb 07 '18 at 16:27
  • My solution in the end to this was to buy a more expensive faster rated CFast 2.0 card! But I should point out the performance is still much worse with many file writes as opposed to writing to the same file. – Russell Newman Mar 06 '18 at 17:48

0 Answers0