6

I'm working with a very large text file (around 70 thousand lines) and I want to remove the top line.

Clearly, loading the entire thing into memory, deleting the top line, then re-writing the whole thing again is inefficient:

var lines = File.ReadLines(accountFileLocation.Text).Skip(1);
File.WriteAllLines("output.txt", lines);

Is there any other way to do it?

user247702
  • 23,641
  • 15
  • 110
  • 157
Jon
  • 2,566
  • 6
  • 32
  • 52
  • Do you have to delete it? How about overwriting it with spaces? 70k lines doesn't seem like much. I'd change it to about 7 trillion lines for impact. – MxLDevs Nov 14 '13 at 04:57
  • The top line will have to turn into the second line, second into third, etc. I want to read from the top line, then never think about it again, even if my program ends abruptly. – Jon Nov 14 '13 at 04:58
  • I don't think there's a way without a full pass through the file. But the implementation should not load the whole file into memory at once. I'll only enumerate as it goes along. Is the performance problematic in your case? How long does the implemetation above take? – Baldrick Nov 14 '13 at 05:00
  • what I think is, use a while loop. before the while loop, do readline() to skip the first line. Then inside the while loop, read line and write line do at the same time. And it seems my idea has been discussed in 'Jon Skeet said : not really' as well. – jhyap Nov 14 '13 at 05:01
  • 2
    possible duplicate of [How to delete a line from a text file in C#?](http://stackoverflow.com/questions/668907/how-to-delete-a-line-from-a-text-file-in-c) – Noctis Nov 14 '13 at 05:03
  • How about changing the file system directly to tell it where the file begins? If a mountain can't be moved, just move the entire island. – MxLDevs Nov 14 '13 at 05:09
  • @Keikoku: Something along these lines? Definitely not worth the pain: http://stackoverflow.com/questions/13430210/insert-bytes-into-middle-of-a-file-in-windows-filesystem-without-reading-entir – Baldrick Nov 14 '13 at 05:40
  • @Baldrick seems like my **jon skeet** comment is still valid, even after reading the post you linked to ... :) (just for the record, jon skeet can do it by looking at the file, and thinking about where he wants to put the bytes ...) – Noctis Nov 14 '13 at 10:25
  • @Noctis: Sacrilege! Jon Skeet does not delete line from files. The lines delete themselves spontaneously out of fear... ;) – Baldrick Nov 14 '13 at 12:54

1 Answers1

1

Hehe .... finally I can say Jon Skeet said :)

Jon Skeet said : not really

What you did is one approach. The second will have to open a read stream and a write stream (to a different file), and read line, then write it into the write if you want it (more for a "test lines for validity, than a all but the first line).

So ... seems like you got the right answer ...

Community
  • 1
  • 1
Noctis
  • 11,507
  • 3
  • 43
  • 82