Java Inputstream too slow to read 1.5mb file

Question

I am working on a part of a project. Its quite big for a newbie like me. In one section I have to fetch an attachment from email and save it to system file. But for some reason, its taking too long time. I used this code for 1st approach:

byte[] buf = new byte[4096];
int bytesRead;
while ((bytesRead = inputStream.read(buf)) != -1) {
    fileOutputStream.write(buf, 0, bytesRead);
}

This code block is taking about 86 secs just for a 1.5Mb file. But when I try to run this same code from a sample test project, It finished in several milliseconds (For this I took the inputstream of a system file instead of attachment and then write it in another directory ). I already visited this pages: Java: InputStream too slow to read huge files

Input stream reads large files very slowly, why?

But I cant find out the specific reason of this. I tried to debug it more by this approach:

InputStream inputStream = bodyPart.getInputStream();
FileOutputStream fileOutputStream = new FileOutputStream(file);
long tStart = System.currentTimeMillis();
byte[] buf = new byte[4096];
int bytesRead;
int i=1;
while(true)
{
    long tc = System.currentTimeMillis();
    bytesRead = inputStream.read(buf);
    long tc1 = System.currentTimeMillis();
    System.out.println("\nRead Time "+i+" ="+(tc1-tc));

    if(bytesRead==-1) break;
    fileOutputStream.write(buf, 0, bytesRead);

    long tc2 = System.currentTimeMillis();
    System.out.println("Write Time "+i+" ="+(tc2-tc1));
    i++;
}

long tEnd = System.currentTimeMillis();
long tDelta = tEnd - tStart;
System.out.println("\n\nTotal Time="+tDelta+"\n\n");

And the output is also weird. A part of output for a 1.5Mb file is:

Read Time 354 =788
Write Time 354 =0

Read Time 355 =0
Write Time 355 =0

Read Time 356 =0
Write Time 356 =0

Read Time 357 =744
Write Time 357 =0

Read Time 358 =0
Write Time 358 =0

Read Time 359 =0
Write Time 359 =0

Read Time 360 =837
Write Time 360 =0

Read Time 361 =0
Write Time 361 =0

Read Time 362 =1
Write Time 362 =0

Read Time 363 =811
Write Time 363 =0

Read Time 364 =1
Write Time 364 =0

Read Time 365 =0
Write Time 365 =0

Read Time 366 =757
Write Time 366 =0

Read Time 367 =1
Write Time 367 =0

Read Time 368 =0
Write Time 368 =0

Read Time 369 =736
Write Time 369 =0

Read Time 370 =0
Write Time 370 =0

Read Time 371 =0
Write Time 371 =0

Read Time 372 =484
Write Time 372 =0

Read Time 373 =0

Total Time=88796

Here, as you can see the read time for 4Kb buffer is taking quite long time after each 3 step or read. I cant figure out the specific issue. I have also used buffered input Stream and output Stream, but the results are same.Can anybody help me to find out what is the actual problem?

Are these attachments stored on a network server ? It's possible that the network server responds slowly - and you cannot do anything about it. — Svetlin Zarev, Jun 06 '16 at 11:56
Actually I downloaded the attachment just above the code i mentioned in the question. So I think it should be in the jvm memory. @SvetlinZarev — For_A_While, Jun 06 '16 at 12:00
If you refer to `bodyPart.getInputStream();`, then it's not in the JVM memory. — Svetlin Zarev, Jun 06 '16 at 12:01
No, I was just running this process to store a attachment. Dont read any other files in parallel. @NicolasFilotto — For_A_While, Jun 06 '16 at 12:05
The total code to fetch attachment is quite large. But the object "bodypart" is the part of the email which is an attachment. The code in my question is running inside a loop for all "bodypart" object which are attachments from an email. @SvetlinZarev — For_A_While, Jun 06 '16 at 12:07
@For_A_While the BodyPart object might be part of the email, but this does not mean that it contains the body itself. It most probably is just a wrapper around a remote call that fetches the data. Put a breakpoint there and step into the `getInputStream()` to see what it;s doing. — Svetlin Zarev, Jun 06 '16 at 12:10
I have run the same code from another simple projects and the code runs fine within several milliseconds. And the code takes same time on my friends pc too. Do I still need to check hard disk activity? @NicolasFilotto — For_A_While, Jun 06 '16 at 12:11
What is `bodyPart.getInputStream`? It is not `InputStream` generically which is "slow" it is the specific `InputStream` you are working with here. What is the source of the data? — Brett Okken, Jun 06 '16 at 12:12
Use a profiler like JProfiler or JVisualVM, and you will know what's wrong — Nicolas Filotto, Jun 06 '16 at 12:21
Obviously, that `bodyPart.getInputStream()` is part of a library for e-mail processing, you didn’t specify. It returns an `InputStream` which encapsulates the processing of an attachment, including, e.g. the Base64 decoding of the attachment. The numbers indicate that this processing is buffered itself, so two of three times, already decoded data is just copied from a buffer to your buffer, the third time, an actual reading and decoding happens to refill the buffer. I guess, it has a 16k buffer holding encoded data, which decodes to 12k actual data which you receive in 3 × 4k read steps. — Holger, Jun 07 '16 at 15:19

score -1 · Answer 1 · answered Jun 06 '16 at 11:50

-1

use bufferedinputstream instead of InputStream.

answered Jun 06 '16 at 11:50

RayanFar

539
11
28

I already said in the question that, I had tried with buffredInputStream and the results are same. – For_A_While Jun 06 '16 at 11:55
1

A `BufferedInputStream` would only potentially make a difference if a larger buffer were being used. The current code uses a 4 KB buffer. – Brett Okken Jun 06 '16 at 12:10
I used bufferdInputStream for 4KB, 8KB, 32KB, 1MB buffer on a 64 bit system. I tried to solve this in different approach. But all took quite similar time. I gave the most common approach of mine in the question. – For_A_While Jun 06 '16 at 12:14
1

A `BufferedInputStream` is no magic. If you look into its source code, you will see, that it does exactly the same thing as the OP’s code. – Holger Jun 07 '16 at 15:06

Java Inputstream too slow to read 1.5mb file

1 Answers1