Why it's needed to open file every time we want to append the file

Question

As the thread How do you append to a file?, most answer is about open a file and append to it, for instance:

def FileSave(content):
    with open(filename, "a") as myfile:
        myfile.write(content)

FileSave("test1 \n")
FileSave("test2 \n")

Why don't we just extract myfile out and only write to it when FileSave is invoked.

global myfile
myfile = open(filename)
def FileSave(content):
    myfile.write(content)

FileSave("test1 \n")
FileSave("test2 \n")

Is the latter code better cause it's open the file only once and write it multiple times? Or, there is no difference cause what's inside python will guarantee the file is opened only once albeit the open method is invoked multiple times.

Answer 1

There are a number of problems with your modified code that aren't really relevant to your question: you open the file in read-only mode, you never close the file, you have a global statement that does nothing…

Let's ignore all of those and just talk about the advantages and disadvantages of opening and closing a file over and over:

• Wastes a bit of time. If you're really unlucky, the file could even just barely keep falling out of the disk cache and waste even more time.
• Ensures that you're always appending to the end of the file, even if some other program is also appending to the same file. (This is pretty important for, e.g., syslog-type logs.)¹
• Ensures that you've flushed your writes to disk at some point, which reduces the chance of lost data if your program crashes or gets killed.
• Ensures that you've flushed your writes to disk as soon as you write them. If you try to open and read the file elsewhere in the same program, or in a different program, or if the end user just opens it in Notepad, you won't be missing the last 1.73KB worth of lines because they're still in a buffer somewhere and won't be written until later.²

So, it's a tradeoff. Often, you want one of those guarantees, and the performance cost isn't a big deal. Sometimes, it is a big deal and the guarantees don't matter. Sometimes, you really need both, so you have to write something complicated where you manually buffer up bits and write-and-flush them all at once.

---

_{1. As the Python docs for open make clear, this will happen anyway on some Unix systems. But not on other Unix systems, and not on Windows..}

_{2. Also, if you have multiple writers, they're all appending a line at a time, rather than appending whenever they happen to flush, which is again pretty important for logfiles.}

Answer 2

In general global should be avoided if possible.

The reason that people use the with command when dealing with files is that it explicitly controls the scope. Once the with operator is done the file is closed and the file variable is discarded.

You can avoid using the with operator but then you must remember to call myfile.close(). Particularly if you're dealing with a lot of files.

One way that avoids using the with block that also avoids using global is

 def filesave(f_obj, string):
     f_obj.write(string)

 f = open(filename, 'a')
 filesave(f, "test1\n")
 filesave(f, "test2\n")
 f.close()

However at this point you'd be better off getting rid of the function and just simply doing:

f = open(filename, 'a')
f.write("test1\n")
f.write("test2\n")
f.close()

At which point you could easily put it within a with block:

with open(filename, 'a') as f:
    f.write("test1\n")
    f.write("test2\n")

So yes. There's no hard reason to not do what you're doing. It's just not very Pythonic.

Answer 3

The latter code may be more efficient, but the former code is safer because it makes sure that the content that each call to FileSave writes to the file gets flushed to the filesystem so that other processes can read the updated content, and by closing the file handle with each call using open as a context manager, you allow other processes a chance to write to the file as well (specifically in Windows).

Answer 4

It really depends on the circumstances, but here are some thoughts:

A with block absolutely guarantees that the file will be closed once the block is exited. Python does not make and weird optimizations for appending files.

In general, globals make your code less modular, and therefore harder to read and maintain. You would think that the original FileSave function is attempting to avoid globals, but it's using the global name filename, so you may as well use a global file altogether at that point, as it will save you some I/O overhead.

A better option would be to avoid globals at all, or to at least use them properly. You really don't need a separate function to wrap file.write, but if it represents something more complex, here is a design suggestion:

def save(file, content):
    print(content, file=file)

def my_thing(filename):
    with open(filename, 'a') as f:
        # do some stuff
        save(f, 'test1')
        # do more stuff
        save(f, 'test2')

if __name__ == '__main__':
    my_thing('myfile.txt')

Notice that when you call the module as a script, a file name defined in the global scope will be passed in to the main routine. However, since the main routine does not reference global variables, you can A) read it easier because it's self contained, and B) test it without having to wonder how to feed it inputs without breaking everything else.

Also, by using print instead of file.write, you avoid having to spend newlines manually.