Write on a file so you keep the last N lines

Question

I am trying to keep a file with only 10 entries.

Every time that I write a new one, the oldest entry is replaced by the newest; so the total lines is always 10.

example of the file:

abcde-12345
akmjk-13249
ofdsi-01230
faaoj-10293
famdk-05931
foajq-10592
xmdsj-19234
boxqa-12130
fdlsp-95392
paidf-19341

When I add a new line with the append command on this file, it goes to increase the total lines of the file, so the file grows every time I add a line. The objective is to append the line at the bottom, removing the first line of the file, so the oldest entry is removed and the new added at the bottom, leaving the total number of lines in the file, always equal to 10.

My approach has been to

-read the file
-save the last 9 lines in a new file
-add the 10th line, which is the new one
-delete the old file, and save the new file with the same name.

This has the problem of being cumbersome, slow, not efficient. In Shell scripting I could do it easily, but since I use python, I hoped for a simpler way to do this

Can you give some code to show a concrete example of your problem? Otherwise its hard to understand your question. With only 10 lines I dont think you should be concerned about consuming resources. — Paul Rooney, May 10 '17 at 22:59
I guess my case is not clear; adding some example code. Thanks — , May 10 '17 at 23:00
combine [How to delete the first line of a text file using Python?](http://stackoverflow.com/questions/20364396/how-to-delete-the-first-line-of-a-text-file-using-python) and [How to append new data onto a new line](http://stackoverflow.com/questions/21839803/how-to-append-new-data-onto-a-new-line) — chickity china chinese chicken, May 10 '17 at 23:03
I need to run this in Python; I could do it in shell scripting, but I would rather see if I can do it in python — , May 10 '17 at 23:07
Your approach (read the whole file, write out the last 9 lines and then write out the 10th line) is the correct way to do it (and it what you're doing in the shell although it may not be obvious). — R Samuel Klatchko, May 10 '17 at 23:28

score 2 · Answer 1 · answered May 10 '17 at 23:13

Use a deque as a buffer, with a maxlen=10, append to the deque, then write from the deque:

Juans-MBP:workspace juan$ cat 10lines.txt
first line
second line
third line
fourth line
fith line
sixth line
seventh line
eigth line
ninth line
tenth line
Juans-MBP:workspace juan$ python
Python 3.5.2 |Anaconda custom (x86_64)| (default, Jul  2 2016, 17:52:12)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import deque
>>> with open('10lines.txt') as f:
...     buffer = deque(f, maxlen=10)
...
>>> buffer.append('first new line\n')
>>> buffer.append('second new line\n')
>>> with open('10lines.txt', 'w') as f:
...     f.writelines(buffer)
...
>>> exit()
Juans-MBP:workspace juan$ cat 10lines.txt
third line
fourth line
fith line
sixth line
seventh line
eigth line
ninth line
tenth line
first new line
second new line
Juans-MBP:workspace juan$

Note, you have to take care of making sure the lines you append have newlines, because as much as the name might imply it, f.writelines doesn't add newlines.

It's worth mentioning that Python3 automatically converts the `\n` to the correct line ending for the platform if the file is opened in text mode. — John La Rooy, May 10 '17 at 23:36

score 0 · Answer 2 · edited May 23 '17 at 12:34

Borrowing from this answer How to delete the first line of a text file using Python? to read and delete the first line, then append the new line and write it back to the file. Here's a possible Python2 solution.

newline = "anoth-48576"

with open('file.txt', 'r') as fin:
    data = fin.read().splitlines(True)

newdata = data[1:]

if not '\n' in newdata[-1]:
    newdata[-1] = newdata[-1]+'\n'

newdata.append(newline+'\n')

with open('file.txt', 'w') as fout:
    fout.writelines(newdata)

Write on a file so you keep the last N lines

2 Answers2