Strip blank lines from a text file

Question

I'm trying to remove blank lines from a text file, so if I have for example:

line 1
line 2

line 3


line 4
line 5

The result after processing the file should look like this:

line 1
line 2
line 3
line 4
line 5

I searched stack overflow and this is what I came out with so far:

with open('file.txt', 'r+', encoding='utf-8') as file:
    for line in file:
        if line.strip():
            file.write(line)

This worked perfectly except for one little inconvenient; instead of replacing the original content with the new one, the result is appended to the end of the file like so:

line 1
line 2

line 3


line 4
line 5line 1
line 2
line 3
line 4
line 5

My question is, is there any way I can fix this issue inside the for loop without external storage (storing the result in a file or list and writing it back to the file)? or if there is a better way please share.

I hope my question was clear and thanks.

Open a second file for output. Then at completion delete first file and rename second to the original name — joel goldstick, Jul 05 '16 at 18:44
See this post: http://stackoverflow.com/questions/2369440/how-to-delete-all-blank-lines-in-the-file-with-the-help-of-python — Jack Evans, Jul 05 '16 at 18:44

score 2 · Answer 1 · answered Jul 05 '16 at 20:15

here is a working example that do what you are looking for. I am not sure about performance here but for sure it does not use the memory or temp file

writebytecount = 0
readbytecount = 0
with open('file2.txt', 'r+') as file:
    while 1:
        line = file.readline()
        if not line: break
        readbytecount = file.tell()
        file.seek(writebytecount)
        file.write(line)
        if len(line.strip()) != 0:
            writebytecount += len(line)+1
        file.seek(readbytecount)
    file.seek(writebytecount)
    file.truncate(writebytecount)
    file.close()

score 1 · Accepted Answer · answered Jul 05 '16 at 18:48

1

So firstly, the output is appended to the file because you open the file in r+ mode and the file pointer is pointing at the end of the file. You won't want to overwrite the file while you're reading it, though, so you really need two files:

with open('infile.txt', 'r', encoding='utf-8') as inFile,\
     open('outfile.txt', 'w', encoding='utf-8') as outFile:
    for line in inFile:
        if line.strip():
            outFile.write(line)

answered Jul 05 '16 at 18:48

Will

24,082
14
97
108

I hoped there was a way to avoid that, but that's OK. Thank you very much. – user3885884 Jul 05 '16 at 18:53
No problem, glad to help! :) You can always overwrite the original file with the newly-created file after writing it. This has the added benefit that if something goes wrong, you can avoid losing data. – Will Jul 05 '16 at 21:00

Strip blank lines from a text file

2 Answers2