How to delete the first line of a BIG file in python efficiently to not waste space?

Question

I have a BIG text file (100 000+ lines) from which I would like to have a loop that always opens the file, gets the first line of text (In this case it is a number), and then deletes that first line, so the next time the file is opened again, the first line is not the same. I have tried to do it so that the line number is always saved in a file and then it iterates to that specific line of the text file, but I believe this is not the most efficient way...

my text file consists of number like this:

1234
2212
3453
1232
... (and it goes on for ages)

The code I have attempted is:

with open('line_number.txt', 'rb') as inp:
line = inp.read()

fp = open("list.txt")

for i, line in enumerate(fp):
    if i == line - 1: #iterates until the number match 

        s = i + 1

        with open('new.txt', 'w') as outp:
            outp.write(s)
        outp.close()

        print(i)

    break
fp.close()

Is there a more efficient way to do this to large files? I believe iterating until the line I need is really time-inefficient...

`readline` will read one line and `read` will read all the rest. — wwii, Sep 27 '20 at 13:25
@wwii maybe I was unspecific, but I would also want to delete the first line after I have gotten the contents of it... From what I researched the ways to do it are not optimal for large files... — George Lua, Sep 27 '20 at 13:27
Quite a few answers searching with `python delete first line from a file in place` pick one and we can mark yours as a duplicate. — wwii, Sep 27 '20 at 13:39

PythonSnek · Accepted Answer · 2020-09-27T13:42:52.850

1

This would work if you need line number 1 to be removed... not if you need to find first occurence of a certain number.

with open('initial_file.txt', 'r') as file:
    lines = file.readlines()
    file.close()
line_that_you_need = lines[0]
with open('initial_file.txt', 'a') as file:
    for i in range(1, len(lines):
        file.write(lines[i])
    file.close

edited Sep 27 '20 at 13:42

answered Sep 27 '20 at 13:27

PythonSnek

542
4
21

`.write()` takes a string for its argument - `lines` is a list. – wwii Sep 27 '20 at 13:41
there, that should work better – PythonSnek Sep 27 '20 at 13:43
Or you could use [https://docs.python.org/3/library/io.html#io.IOBase.writelines](https://docs.python.org/3/library/io.html#io.IOBase.writelines) – wwii Sep 27 '20 at 13:45

How to delete the first line of a BIG file in python efficiently to not waste space?

1 Answers1