0

I noticed that when writing to files in Python, the contents are written to the file once it is closed.

For example:

with open('test.txt','w') as ffile:
    ffile.write("testing")
    time.sleep(20)

In this example "testing" will be written after waiting 20 seconds.

How and why does this happen? Why not just writing to the file simultaneously?

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Eleftheria Chatziargyriou
  • 236
  • 2
  • 8

2 Answers2

5

You are seeing the effects of buffering. Disk I/O uses buffers to improve performance, and you have not written enough data to the buffer for it to flush.

Write more data, or close the file, both cause the buffer to be flushed. Alternatively, set the buffer size to a very small number (the number of bytes the buffer will hold):

with open('test.txt', 'w', 2) as ffile:

The options 0 and 1 have special meaning; 0 would disable buffering altogether (only available for binary mode files) and 1 is the default for text files (using line buffering, write a newline to flush).

That also means that if you have a text file, you could write a newline to trigger flush:

ffile.write('\n')

Last but not least, you could flush explicitly by using the file.flush() method:

ffile.flush()
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Thank you,can you elaborate more on how the performance is improved, or at least link to a source? – Eleftheria Chatziargyriou Jan 13 '17 at 15:59
  • 1
    Maybe also explain what `2` does. The option you are using. – Ma0 Jan 13 '17 at 16:00
  • @Ria: see the [general info on data buffers](https://en.wikipedia.org/wiki/Data_buffer). I/O to a physical medium is *slow*, but has a large bandwidth. You can move a large amount of data to a file, but it takes a while to get there. You don't want your program to pay that waiting price for each individual write; better to collect data into a larger buffer and move stuff to the file in chunks. – Martijn Pieters Jan 13 '17 at 16:02
  • @MartijnPieters Is that why `file.write()` does not append a newline by defualt? Because it triggers flushing? And another thing, if one uses `print('something', file='my_file')` would it flush because `print` appends a newline? – Ma0 Jan 13 '17 at 16:03
  • @Ev.Kounis: Why would you want the file to decide that you need a newline every time you write? Only `print()` will do that for you; use `print(.., file=fileobject)` if you want to write to a file and have newlines added automatically. – Martijn Pieters Jan 13 '17 at 16:04
  • Thank you, one last question: Is this the reason why printing to console is slower than directly writing to a file? – Eleftheria Chatziargyriou Jan 13 '17 at 16:04
  • @Ev.Kounis: yes, if the file is configured for line buffering (the default for text files), that'll trigger a flush. – Martijn Pieters Jan 13 '17 at 16:05
  • @Ria: Printing to the console is not slower or faster than a file object. It depends on the console; is it running a large scroll-back buffer that takes a lot of memory? Is it doing a lot of line width redrawing? A console is not a file, it can be a complex piece of software that renders to your screen. – Martijn Pieters Jan 13 '17 at 16:06
1
with open('test.txt','w') as ffile:
    ffile.write("testing")
    ffile.flush() # This forces the buffer to be written to the file.
    time.sleep(20)
gipsy
  • 3,859
  • 1
  • 13
  • 21
  • syntactically correct, but offers little explanation of why OP observes _contents are written to the file once it is closed_. Can you add that to your answer? (even a comment after the `flush` statement would be an improvement). – ryyker Jan 13 '17 at 17:18
  • @ryyker Thanks. Updated my answer. – gipsy Jan 13 '17 at 17:20