5

I would like to know what is the fastest and more efficient way of deleting last N lines of a .txt file.

I have been looking for different previous questions and I found that reading and copying lines again is one way.

Remove lines from textfile with python

Nonetheless, I have been trying to look for other way such as skiprows or skipfooter on pandas and be able to save the outcome to other txt.

At the same time, I have seen threads about "head" and "sed" but not sure about their usage in Python.

Could you, please, give advice on this topic?

FJSJ
  • 219
  • 3
  • 14
  • How would you know which lines are the last N lines without reading it? (If you can, you can just read the lines you need) – BlueSheepToken Jan 28 '19 at 10:16
  • 1
    In here, you have pretty low level answer how to remove last line, maybe it will help you gain proper performance results with more than just one, final line :) https://stackoverflow.com/a/10289740/5407075 – Damian Jan 28 '19 at 10:23
  • Thank you for your answers BlueSheepToekn and Damian! – FJSJ Jan 28 '19 at 10:59

1 Answers1

6

if you want to skip last n lines using pandas then:

import pandas as pd

df = pd.read_csv('yourfile.txt', skipfooter = N)
df.to_csv('yournewfile.txt')

Change the delimiter using sep = ... if necessary

Alternatively do as per the answer you cited, but using with open(..) as good practice:

with open('yourfile.txt') as f1:
    lines = f1.readlines()

with open('yournewfile.txt', 'w') as f2:
    f2.writelines(lines[:-N])
jambannabelbar
  • 86
  • 5
  • 1
    First option is the one I was looking for because It is so easy to avoid the last N lines. Then just need to save the read to other file. Thanks! – FJSJ Jan 28 '19 at 12:34
  • this method can be very time consuming on big files. Writing the whole file again can take very long time – sagi Jun 01 '20 at 14:21