Java - File To Byte Array - Fast One

Question

I want to read a file into a byte array. So, I am reading it using:

    int len1 = (int)(new File(filename).length());
    FileInputStream fis1 = new FileInputStream(filename);
    byte buf1[] = new byte[len1];
    fis1.read(buf1);

However, it is realy very slow. Can anyone inform me a very fast approach (possibly best one) to read a file into byte array. I can use java library also if needed.

Edit: Is there any benchmark which one is faster (including library approach).

How slow is it? Which part of it is slow? I for one am not aware of it being very slow. — Marko Topolnik, Jul 31 '12 at 13:57
Slow as compared to *what*? Are you sure it's just not a case of percieved slowness in which case the solution is to try not to read the entire file in a single go and think of processing it chunk by chunk? — Sanjay T. Sharma, Jul 31 '12 at 13:59
@MarkoTopolnik, Takes 15-20 sec to read 1616007480 bytes. Last line is slow :). — alessandro, Jul 31 '12 at 14:00
@SanjayT.Sharma, I have to process whole file, not chunk by chunk. — alessandro, Jul 31 '12 at 14:02
@alessandro: It's normal for a 1.5 GB to take noticeable amount of time to be read completely in memory, you just can't make stuff magically faster. :) — Sanjay T. Sharma, Jul 31 '12 at 14:02
See [this question](http://stackoverflow.com/questions/858980/file-to-byte-in-java) — elias, Jul 31 '12 at 14:03
Let's see... 1.5GB in 15 seconds... WOW, that's 100MB/s! That's **FAST**! — Marko Topolnik, Jul 31 '12 at 14:08

score 16 · Accepted Answer · answered Jul 31 '12 at 14:03

It is not very slow, at least there is not way to make it faster. BUT it is wrong. If file is big enough the method read() will not return all bytes from fist call. This method returns number of bytes it managed to read as return value.

The right way is to call this method in loop:

  public static void copy(InputStream input,
      OutputStream output,
      int bufferSize)
      throws IOException {
    byte[] buf = new byte[bufferSize];
    int bytesRead = input.read(buf);
    while (bytesRead != -1) {
      output.write(buf, 0, bytesRead);
      bytesRead = input.read(buf);
    }
    output.flush();
  }

call this as following:

ByteArrayOutputStream baos = new ByteArrayOutputStream();
copy(new FileInputStream(myfile), baos);
byte[] bytes = baos.toByteArray();

Something like this is implemented in a lot of packages, e.g. FileUtils.readFileToByteArray() mentioned by @Andrey Borisov (+1)

EDIT

I think that reason for slowness in your case is the fact that you create so huge array. Are you sure you really need it? Try to re-think your design. I believe that you do not have to read this file into array and can process data incrementally.

Note how you pass 2 parameters into copy method, which has 3 parameters. — divanov, May 08 '13 at 13:27

score 10 · Answer 2 · answered Jul 31 '12 at 13:58

10

apache commons-io FileUtils.readFileToByteArray

answered Jul 31 '12 at 13:58

Andrey Borisov

3,160
18
18

1

internally this method invokes copyLarge(InputStream input, OutputStream output) - so should be better =:) Also apache commons-io uses it's own ByteArrayOutputSteam. from docs ---> This is an alternative implementation of the java.io.ByteArrayOutputStream class. The original implementation only allocates 32 bytes at the beginning. As this class is designed for heavy duty it starts at 1024 bytes. In contrast to the original it doesn't reallocate the whole memory block but allocates additional buffers. This way no buffers need to be garbage collected... – Andrey Borisov Jul 31 '12 at 14:00

Java - File To Byte Array - Fast One

2 Answers2

Linked