Java windows write bytes to file, some are incorrect

Question

My goal is to read a stream of bytes from the socket into a file, and then play it back at a later time as test harness for my application. Somewhere in writing bytes to disk, a byte will get written incorrectly, seemingly at random.

My writer looks like this:

blobWriter = new BufferedOutputStream(new FileOutputStream(blobFileName));  
blobChannel = Channels.newChannel(blobWriter);

I'm using a blobChannel so that I can write directly from a ByteBuffer. On each read of the socket, I simply pass the buffer to the writer:

if (key.isReadable()) {
    final int bytesRead= socketChannel.read(readBuffer);

    if(bytesRead == -1)
    {
        logger.warn("no bytes to read");
        break;
    }

    readBuffer.flip();
    blobChannel.write(readBuffer);
    ... 
    <continue to process data>
}

When the feed is live, the program processes reads into records, and they are not corrupt. Say for each message, it outputs a tuple of 7 fields. One of them, for example, is this:

(tupleid=0,msgType=110,feedId=225,venueId=30,orderId=160,symbol="CHF.NOK.SPOT",venueTime=44417979)

When instead of a live connection to the market, I hook the application to a reader that plays the same data back from disk, the processed output goes haywire:

(tupleid=0,msgType=110,feedId=225,venueId=30,orderId=160,symbol="CHF.-�ûnX",venueTime=44417979)

Notice the corrupt symbol.

The weirdest thing is that it will process thousands of messages with the same symbol and other fields no problem, but then inexplicably one message gets corrupted. It's not always the symbol field that is incorrect, sometimes the orderId is wrong etc...

I suspect that blobWriter is miswriting on occasion. Could my OS (windows 7) is doing something funky? I've inspected the bytestream that is saved to disk in notepad++, and indeed it shows the incorrect bytes, so the error must be in the file writer, not in my playback mechanism. Furthermore, if the main application itself was buggy, it should misread bytes on the live feed; it doesn't.

Does anyone know what could possibly be going wrong?

Answer 1

Looks like the WritableByteChannel javaDoc was trying to warn me:

Unless otherwise specified, a write operation will return only after writing all of the r requested bytes. Some types of channels, depending upon their state, may write only some of the bytes or possibly none at all. A socket channel in non-blocking mode, for example, cannot write any more bytes than are free in the socket's output buffer.

Indeed the socket channel is in non-blocking mode. I profiled this and one in every 200 reads or so is out of sync. It seems like my channel is writing more bytes that my readers is actually reading from buffer.

Another SO thread offers more information.

What finally worked for me was copying the bytearray on each read instead of using a stream. This is probably not performance optimal, but at least it doesn't result in wrong data:

if (readBuffer.hasRemaining()){
    byte[] b = new byte[readBuffer.remaining()]
    readBuffer.get(b)
    blobWriter.write(b)
}