Replacing a line on Python

Question

I'm trying to convert PHP code to Python, and I have problems with replacing lines. Although I find it easier to do using Python, I'm absolutely lost; I can find the line to replace, I can add something to the end of the line, but I can't write the line again on the file.

file = open("cache.ucb", 'rb')
for line in file:
   if line.split('~!')[0] == ex[4]:
       line += "~!" + mask[0]
       line = line.rstrip() + "\n"
       # Write on the file here!

Basically, the file uses ~! as a separator, and I read each line. If the first token separated with ~! of the line starts with ex[4], which could be for example Catbuntu, I want to append mask[0], which could be Bousie, on the end of that line. Then I remove the new line characters and add one to the end.

And there's the problem. I want to write the file as it was, but changing only that line. Is that possible?

Answer 1

Assuming you're on python >=2.7, the following should work a treat

original = open(filename)
newfile = []
for line in original:
    if line.split('~!')[0] == ex[4]:
        line += "~!" + mask[0]
        line = line.rstrip() + "\n"
    newfile.append(line)
original.close()
amended.open(filename, "w")
amended.writeLines(newfile)
amended.close()

If for whatever reason you are on python 2.6 or lower, replace the second to last line with:

amended.write("".join(newfile))

EDIT: Fixed to replace a mistake copied from the question, factor out a filename.

Answer 2

You cannot modify a file in-place, at least not if you want to insert characters to a line. You'll just end up overwriting the start of the next line.

There are two different ways to do this:

1. Read the file into memory, close it, then write back the new version.
2. Write a new temporary file as you go along, then move it over the original version.

So, how do you choose between them? I'll try to summarize the differences, ordered so that each one typically trumps the ones below if it's important (but that's just "typically"—you have to think through your own use case):

• 2 doesn't require holding the entire thing in memory. If your file is, say, 20GB long, this is obviously a huge win; if it's 16KB, it doesn't matter.

• 2 makes the entire operation atomic. Even if it fails halfway through, or some other process tries to read the file while you're in the middle of changing it, there is no way anyone can see some invalid half-modified file; they will see either the original file, or the new one.

• 2 requires some free disk space (because there are, temporarily, two copies of the file at the same time).

• 2 is a huge pain in the neck if you care about both Windows and POSIX.

• 2 can involve copying across filesystems if the original file and the temp directory are on different filesystems, unless you're careful about it.

• 2 is simpler if neither of the above two are an issue.

Drakekin's answer tells you how to do #1.

Here's how to do #2 if you don't care about Windows or about cross-filesystem issues:

infile = open("cache.ucb", 'rb')
outfile = tempfile.NamedTemporaryFile(delete=False)
for line in infile:
   if line.split('~!')[0] == ex[4]:
       line += "~!" + mask[0]
       line = line.rstrip() + "\n"
   outfile.write(line)
infile.close()
os.rename(outfile.name, "cache.ucb")
outfile.close()

You can solve the cross-filesystem problem by, e.g., passing dir=os.path.dirname(original path) to the NamedTemporaryFile constructor, but only if you're sure you'll always have permissions to create a new file alongside the original (which isn't always guaranteed, just because you have permission to rewrite the original—UNIX permissions, Windows ACLs, the OS X sandbox, etc. all give ways that can be false).

To solve the Windows problem… well, start with Is an atomic file rename (with overwrite) possible on Windows, and similar discussions all over the internet.

Answer 3

Open the file in mode 'wb' and put file.write(line) at the end of your loop.

Answer 4

You don't have your file open for writing.

file = open("cache.ucb", 'rb')

This line opens a file for reading in binary mode. You need to open it for writing also.

Try opening the file in write mode, 'w' and writing the line back.

Or you can simply open your file for read/write at the beginning and write inside your loop:

file = open("cache.ucb", 'a+')