1

I am writing a bit torrent client and allocate space on hard drive for each file I have to download.

File can be huge and I want to overwrite some piece of it without deleting all content.

I read some other answers, but they suggest creating a temporary space and then copying changes back. That would be too heavy process.

I am sure there must be some utility.

Burhan Khalid
  • 169,990
  • 18
  • 245
  • 284
8bra1nz
  • 717
  • 7
  • 19
  • It is very difficult to modify in place binary files. I think this is what you are trying to do. For example, if you are downloading a mp3 file or a movie, it is very difficult - almost impossible - to overwrite a specific part of the file, without corrupting the rest of it. If you are downloading a plain text file, then this is a bit simpler - however, it is more efficient to use an intermediary temporary file. – Burhan Khalid Apr 24 '16 at 05:25
  • @BurhanKhalid Exactly. – 8bra1nz Apr 24 '16 at 05:26
  • What is the exact problem you are facing? – Burhan Khalid Apr 24 '16 at 05:27
  • I need to write file piece by piece and I don't want to have whole file content on RAM. I cant find utility to do it without overwriting file with same content each time every 32 kb piece gets downloaded. I can preallocate enough space. – 8bra1nz Apr 24 '16 at 05:29
  • And pieces are not downloaded in order. – 8bra1nz Apr 24 '16 at 05:30
  • Are you creating a streaming client? This is the only scenario where the order of chunks is important. – Burhan Khalid Apr 24 '16 at 05:35
  • I am writing bittorrent client. – 8bra1nz Apr 24 '16 at 05:37
  • Most languages should provide one of the following (since that's what provided by posix standards): seek+write buffer of a given length, write with offset + length, memory-map a region of that file, write to mapping. You'll have to make sure that the file is opened for writing, not for append. All of those should write just the data without clobbering anything else. – the8472 Apr 30 '16 at 14:57

2 Answers2

1

Since you are talking about a download file, you should consider the storage of downloaded data in a temporary file(chunks). Refer : streaming & text-file

If I read you question (without python), I would say you could use OS specific commands to overwrite parts of a file.

For example in Linux, use sed or awk. Refer : Inserting-a-line-in-a-file-at-a-specific-location

Community
  • 1
  • 1
Ani Menon
  • 27,209
  • 16
  • 105
  • 126
0

I found a solution, using OS (Linux) specific system call:

fd = os.open(path, os.O_WRONLY)
os.lseek(fd, offset, os.SEEK_SET)
os.write(fd, piece)
os.close(fd)

This did overwrite file without nullifying it. But when had:

with open(path, 'wb') as file:
    file.seek(offset)
    file.write(piece)

It was deleting all other content.

Program is not portable this way but I don't need it to be.

8bra1nz
  • 717
  • 7
  • 19