0

I am trying to divide a big file using Java. Is there a way to divide a file and create smaller chunks without reading the entire file? I can calculate the points in the file where it should be splitted.

Does anyone know how the split command(unix) does it?

user6176171
  • 1
  • 2
  • I don't think it is possible to do so without reading each line in the file at least once. See the link posted by @reto for a good solution. – SamTebbs33 Apr 08 '16 at 08:47
  • @SamTebbs33 yes I know the solution to read the entire file, thats what I am trying to avoid. Does anyone knows if the split command reads the entire file to split it? – user6176171 Apr 08 '16 at 08:51

1 Answers1

0

You need RandomAccessFile, it can reads file by bytes , so you could do stream-like processing.

Thought RandomAccessFile is quite easy to use, but it turns out pretty slow for big files. The reason is, it needs to interactive with disk every time it reads something, that will make a lot of data copies.

If you need to speed it up, you can use MappedByteBuffer and FileChannel as an alternative.

WoooHaaaa
  • 19,732
  • 32
  • 90
  • 138