4

Lets say I have a modestly sized text file (~850kb, 10,000+ lines)

And I want to replace a particular line (or several) spread out amongst the file.

Current methods for doing this include re-writing the whole file. The current method I use is read through the entire file line by line, writing to a .tmp file, and once I am done, I rename() the tmp file to the original source file.

It works, but it is slow. And of course, as the file grows, so will execution times.

Is there another way (using PHP) to get the job done without having to rewrite the entire file every time a line or two need to be replaced or removed?

Thanks! I looked around and could not find an answer to this on stackoverflow.

Kovo
  • 1,687
  • 14
  • 19
  • 2
    You have to loop the entire file. http://stackoverflow.com/questions/235604/overwrite-line-in-file-with-php – Ron van der Heijden Apr 05 '12 at 14:47
  • So there is no other way from the method I currently employ? – Kovo Apr 05 '12 at 14:49
  • Assume that each row is a record, that may be changed or become irrelevant at some point. Sqlite and a database are not options (or else this post would not exist!) – Kovo Apr 05 '12 at 14:56
  • I know it won't help your question, but why isn't a database an option? It would really make things a helluvalot easier. – CodeCaster Apr 05 '12 at 14:57

2 Answers2

6

If the replacement is EXACTLY the same size as the original line, then you can simply fwrite() at that location in the file and all's well. But if it's a different length (shorter OR longer), you will have to rewrite the portion of the file that comes AFTER the replacement.

There is no way around this. Shorter 'new' lines will leave a gap in the file, and longer 'new' lines would overwrite the first part of the next line.

Basically, you're asking if it's possible to insert a piece of wood in the middle of another board without having to move the original board around.

Marc B
  • 356,200
  • 43
  • 426
  • 500
  • That is what I was afraid of. Of course a databse would solve this, but in this particular case, a databse or sqlite is not possible. Thanks! – Kovo Apr 05 '12 at 14:58
  • 2
    Actually, you should be able to always skip the offset to the change. Apart from that, block copying is not possible with PHP AFAIK, so for the rest of the file I suggest you take some buffer (instead of doing this line by line). I think the [`SPLFileObject`](http://php.net/SPLFileObject) does the buffering internally while giving your line-by-line access to the file, so this might be the best you can get with less code. – hakre Apr 05 '12 at 15:50
3

You can't, because of the way files are stored on common filesystems. A file always takes up one or more 'blocks' of disk space, where blocks are for example 4096 bytes in size. A file that has one byte of data, will still occupy one whole block (consuming 4096 bytes of available disk space), while a file of 4097 bytes will occupy two blocks (taking up 8192 bytes).

If you remove one byte from a file, there will be a gap of one byte inside one of the blocks it occupies, which is not possible to store on disk. You have to shift all other bytes one byte to the beginning of the file, which will affect the current and all following blocks.

The other way around, adding bytes in the middle of a block, shows the same problem: you'll have one or more bytes that don't fit in the 4096 bytes anymore, so they'll have to shift into the next block, and so on, until the end of the file (and all blocks) has been reached.

The only place where you can have non-occupied bytes in a block, is at the end of the last block that forms a file.

CodeCaster
  • 147,647
  • 23
  • 218
  • 272