0

I am currently working with data cleaning. I need to clean a big csv file with a few head rows and the last row need to be chopped off. Is there any way I can just chopped the rows without load the whole file?

V.Yan
  • 109
  • 7
  • 2
    `Is there any way` <- Yes there is. However, without knowing what's deficient about your solution, we're not going to be able to help you come up with a better solution – inspectorG4dget Oct 12 '17 at 20:18
  • You could do it through bash. Maybe see [last line](https://stackoverflow.com/questions/4881930/remove-the-last-line-from-a-file-in-bash) / [first line](https://stackoverflow.com/questions/339483/how-can-i-remove-the-first-line-of-a-text-file-using-bash-sed-script) – Mr.cysl Oct 12 '17 at 20:18

2 Answers2

0

Grabbed this from another similar thread, it should resolve your issue:

for line in reversed(open("filename").readlines()):
    print line.rstrip()

And in Python 3:

for line in reversed(list(open("filename"))):
    print(line.rstrip())
Austin A
  • 566
  • 2
  • 15
  • Note: this does load the whole file. I'm not sure of any way to not do that, so maybe someone else can provide a better solution. – Austin A Oct 12 '17 at 20:41
0

Python is line oriented, right down into the interpreter.
I think that the best way to do this would be to sys.call and using an external program that is better suited for file level manipulations. This is not cheating, python is a great scripting language. You should use the unix tools.

Remove lines from file with head, tail, or sed

VoNWooDSoN
  • 1,173
  • 9
  • 13