0

I have a log file with 12_500_000 lines.

I would like to remove the first n lines from the file, ideally not having to create a copy.

The problem with writing changes to a new file after removing every line is the time and space complexity.

Any ideas on how to do that?

kryzystof
  • 190
  • 2
  • 13
  • Hi @kryzystof, this might help you https://stackoverflow.com/questions/1377279/find-a-line-in-a-file-and-remove-it – Dren Mar 14 '21 at 22:33
  • 2
    If you're trying to do that for performance reasons: don't. Text files are not built for efficient insertion/removal of lines. You'll effectively have to rewrite the file with the first n lines skipped which means you'll have to read the whole file and write all the lines you want to keep. It also smells like an [XY Problem](https://xyproblem.info). Specifically you might want to look into log rotation, because that is pretty much a solved problem, please don't attempt to custom-build your own solution for this. – Joachim Sauer Mar 14 '21 at 22:35
  • Be aware that any scheme for removing lines from a file will (at least) rewrite the entire the rest of the file. Writing to a separate file is safer. Updating the file in place risks corrupting the file ... if something goes wrong while you are doing the operation. (For your use-case, the unsafe approach doesn't alter big-O time complexity, but you do save temporary file space usage during the operation.) – Stephen C Mar 14 '21 at 23:07

0 Answers0