BufferedInputStream to ByteArrayOutputStream very slow

Question

I have a problem very similar to the link below:

The main difference being I am trying to interpret a Socket connection via a ServerSocket containing Binary, rather than a file. This works as expected.

However, the problem I am having is that this process is taking quite a long time to read into memory, about 1 minute 30 seconds for 500 bytes (although the size of each stream will vary massively)

Here's my code:

BufferedInputStream input = new BufferedInputStream(theSocket.getInputStream());
byte[] buffer = new byte[8192];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();

while ((bytesRead = input.read(buffer)) != -1)
{
    output.write(buffer, 0, bytesRead);
}

byte[] outputBytes = output.toByteArray();

//Continue ... and eventually close inputstream

If I log it's progress within the while loop within the terminal it seems to log all the bytes quite quickly (i.e. reaches the end of the stream), but then seems to pause for a time before breaking out of the while loop and continuing.

Hope that makes sense.

Is the server closing the connection? What you describe looks like the stream is blocking, waiting for more bytes to arrive. — Norwæ, Feb 25 '14 at 12:03

score 5 · Accepted Answer · answered Feb 25 '14 at 12:03

5

Well you're reading until the socket is closed, basically - that's when read will return -1.

So my guess is that the other end of the connection is holding it open for 90 seconds before closing it. Fix that, and you'll fix your problem.

answered Feb 25 '14 at 12:03

Jon Skeet

1,421,763
867
9,128
9,194

1

And if you have no control over the other end of connection, you can try setting a timeout for the socket: `theSocket.setSoTimeout(10000); //10s socket timeout` – Vlad Feb 25 '14 at 12:13
That makes sense. I assumed that read -1 meant all information had been received, not that the socket had timed out. My problem is there is no way to tell when the 'end of socket stream' is, i.e. there is no fixed size or byte at the end of the stream to indicate the 'end' – D4y Feb 25 '14 at 12:25
@D4y: Exactly - it's just a stream of data. If you want a message-based protocol (e.g. via length-prefixing or end-of-message data) you need to put that in yourself. – Jon Skeet Feb 25 '14 at 12:27
Yeah, problem is I don't control the contents of the message and need the connection to stay open as once everything is receive and processed I send a response back via the socket using an output stream. – D4y Feb 25 '14 at 12:32
@D4y: Then I'm afraid your protocol is fundamentally broken, unless you can spot the end of the message by the content itself. – Jon Skeet Feb 25 '14 at 12:32
There is a predefined 'end' to the stream but its not a simple string I can detect easily (its some characters with a timestamp wrapped within them) I guess I can just detect when this pattern occurs and continue until I reach that. Thanks for the help!! – D4y Feb 25 '14 at 12:59

score 0 · Answer 2 · answered Feb 25 '14 at 12:18

0

ByteArrayOutputStream(int size);

By default the size is 32 bytes so it increses like this: 32->64->128->256->... So initialize it with a bigger capacity.

answered Feb 25 '14 at 12:18

Athanor

855
1
16
34

score 0 · Answer 3 · answered Feb 25 '14 at 12:26

You can time how long it takes to copy data between a BufferedInputStream and a ByteArrayOutputStream.

int size = 256 << 20; // 256 MB
ByteArrayInputStream bais = new ByteArrayInputStream(new byte[size]);
long start = System.nanoTime();
BufferedInputStream input = new BufferedInputStream(bais);
byte[] buffer = new byte[8192];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();

while ((bytesRead = input.read(buffer)) != -1) {
    output.write(buffer, 0, bytesRead);
}

byte[] outputBytes = output.toByteArray();
long time = System.nanoTime() - start;
System.out.printf("Took %.3f seconds to copy %,d MB %n", time / 1e9, size >> 20);

prints

Took 0.365 seconds to copy 256 MB

It will be much faster for smaller messages i.e. << 256 MB.

BufferedInputStream to ByteArrayOutputStream very slow

3 Answers3