Adding content to middle of file..without reading it till the end

Question

I have read various questions/answers here on unix.stackexchange on how to add or remove lines to/from a file without needing to create a temporary file.

https://unix.stackexchange.com/questions/11067/is-there-a-way-to-modify-a-file-in-place?lq=1

It appears all these answers need one to atleast read till end of the file, which can be time consuming if input is a large file. Is there a way around this? I would expect a file system to be implemented like a linked list...so there should be a way to reach the required "lines" and then just add stuff (node in linked lists). How do I go about doing this?

Am I correct in thinking so? Or Am I missing anything?

Ps: I need this to be done in 'C' and cannot use any shell commands.

Files are not implemented as linked lists, so there isn't a simple way to insert data into the middle of a file. See [SO 10467711](http://stackoverflow.com/questions/10467711) for code to handle inserting data into the middle of a file, but be aware that it does end up copying all the data that follows the insertion point to the correct new position. — Jonathan Leffler, Jun 20 '13 at 00:34

score 11 · Answer 1 · answered Oct 26 '15 at 05:31

As of Linux 4.1, fallocate(2) supports the FALLOC_FL_INSERT_RANGE flag, which allows one to insert a hole of a given length in the middle of a file without rewriting the following data. However, it is rather limited: the hole must be inserted at a filesystem block boundary, and the size of the inserted hole must be a multiple of the filesystem block size. Additionally, in 4.1, this feature was only supported by the XFS filesystem, with Ext4 support added in 4.2.

For all other cases, it is still necessary to rewrite the rest of the file, as indicated in other answers.

Thank you ! Does your answer fit my question : http://unix.stackexchange.com/q/281652/9689 ? I've quoted it there, but if you add it as your own answer I will be happy to mark it as answer and remove my quote. :) — Grzegorz Wierzowiecki, May 07 '16 at 19:54

score 9 · Answer 2 · answered Jun 19 '13 at 18:15

The short answer is that yes, it is possible to modify the contents of a file in place, but no, it is not possible to remove or add content in the middle of the file.

UNIX filesystems are implemented using an inode pointer structure which points to whole blocks of data. Each line of a text file does not "know" about its relationship to the previous or next line, they are simply adjacent to each other within the block. To add content between those two lines would require all of the following content to be shifted further "down" within the block, pushing some data into the next block, which in turn would have to shift into the following block, etc.

In C you can fopen a file for update and read its contents, and overwrite some of the contents, but I do not believe there is (even theoretically) any way to insert new data in the middle, or delete data (except to overwrite it with nulls.

Thanks, What you said is correct..but unfortunately I can only accept one answer. So I am leaving at upvoting. — user763410, Jun 28 '13 at 19:55
Don't filesystems allow for fragmentation? For instance, even ext4 fragments under certain circumstances. — Mateen Ulhaq, Apr 22 '18 at 00:03

score 4 · Accepted Answer · answered Jun 19 '13 at 18:33

You can modify a file in place, for example using dd.

$ echo Hello world, how are you today? > helloworld.txt
$ cat helloworld.txt
Hello world, how are you today?
$ echo -n Earth | dd of=helloworld.txt conv=notrunc bs=1 seek=6
$ cat helloworld.txt
Hello Earth, how are you today?

The problem is that if your change also changes the length, it will not quite work correctly:

$ echo -n Joe | dd of=helloworld.txt conv=notrunc bs=1 seek=6
Hello Joeth, how are you today?
$ echo -n Santa Claus | dd of=helloworld.txt conv=notrunc bs=1 seek=6
Hello Santa Clausare you today?

When you change the length, you have to re-write the file, if not completely then starting at the point of change you make.

In C this is the same as with dd. You open the file, you seek, and you write.

score 0 · Answer 4 · answered Aug 18 '14 at 16:24

You can open a file in read/write mode. You read the file (or use "seek" to jump to the position you want, if you know it), and then write into the file, but you overwrite the data which are here (this is not an insertion). Then you can choose to trunc the file from the last point you wrote or keep all the remaining data after the point you wrote without reading them.

Adding content to middle of file..without reading it till the end

4 Answers4

Linked