Joining large files into one humongous file

Question

Is there a way in Windows to link multiple files together without having to open the target file and read the contents of the source files to append them to the target file? Something like a shell link api?

Background

I have up to 8 seperate processes creating parts of a data file that I want to recombine into one large file.

This goes well with [a question](http://stackoverflow.com/questions/5560191/prepending-data-to-a-file) I just asked ~3 minutes ago I think. :) — user541686, Apr 06 '11 at 00:34
This is not a programming-related question and should be moved to another forum. — Steven, Apr 07 '11 at 06:31
it is a programming issue if there is an api for joining files that can be called from python or c. — Martlark, Apr 07 '11 at 12:27

score 1 · Answer 1 · edited Apr 12 '11 at 19:02

1

No simple way that I know of. But here's a radical idea.

Use a virtual file system (Dokan, EldoS CBFS, Pismo Technic, etc..) to emulate one logical file that is actually backed by separate files on disk.

edited Apr 12 '11 at 19:02

Eugene Mayevski 'Callback

45,135
8
71
121

answered Apr 06 '11 at 05:52

selbie

100,020
15
103
173

If only my organisation was open to radical ideas! – Martlark Apr 07 '11 at 12:27

score 1 · Accepted Answer · answered Apr 07 '11 at 22:00

1

A less radical solution that should work just fine.

system("copy filefragment.1+filefragmenent.2+filefragment.3+....+filefragment.8 outputfile.bin");

answered Apr 07 '11 at 22:00

selbie

100,020
15
103
173

This is what I went with. Feels like cheating though. – Martlark Apr 09 '11 at 23:38

score 0 · Answer 3 · edited Nov 06 '12 at 06:44

I have up to 8 seperate processes creating parts of a data file that I want to recombine into one large file.

How do you want them concatenated? Mixed or one after the other?

If you want them mixed, you can just open() your output file and write() to it from your threads. If you want them one after the other, you're best bet is to write to separate files and join them together at the end.

Joining large files into one humongous file

3 Answers3