1

I have a program constantly writing to a file, which is compressed on the fly:

import lzma

with lzma.open('file.lz', 'wb') as f:
    for ...:
        # do something
        f.write(item)

So, the file is append-only. At the same time, I need to be able to run another program which will read from this file - not streaming/following, just one-shot reading of the current content. Basically, this works:

import lzma

with lzma.open('file.lz', 'rb') as f:
    content = f.read()

But writes in the first program don't write to the file immediately, and instead they buffer the data for some time (I see buffers of the size 8k to 60k). When small writes happen infrequently the file content can become very far from the current state and I'd like to flush it or do something similar (each n records or each n minutes). However, f.flush() doesn't seem to do anything. What's the best solution here, maybe I overlooked something obvious?

aplavin
  • 2,199
  • 5
  • 32
  • 53
  • According to [this answer](https://stackoverflow.com/a/3168436/5276734) you can set the buffer to zero for normal file `open()`, but for `lzma.open()` this seems not available. – bastelflp Nov 05 '17 at 16:42
  • @bastelflp for normal non-compressed `open` there is no problem with `flush`ing when needed. – aplavin Nov 05 '17 at 16:45

0 Answers0