0

I'm using ByteArrayRawSerializer as a socket message deserializer. The message end is always indicated by the server closing the socket.

As messages may be large, I'd like to define the messagesize of the serializer unlimited. But how?

The following leads to a buffer overflow error:

ByteArrayRawSerializer s = new ByteArrayRawSerializer();
s.setMaxMessageSize(Integer.MAX_VALUE);
membersound
  • 81,582
  • 193
  • 585
  • 1,120

1 Answers1

1

It is completely impractical to use such a huge buffer size, each new request would try to allocate > 2Gb of memory.

You need to use a more reasonable size that is large enough to handle your expected message sizes.

Or, create a custom deserializer that allocates more buffers as needed.

EDIT

Here's an elastic raw deserializer...

/*
 * Copyright 2017 the original author or authors.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.springframework.integration.ip.tcp.serializer;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;

import org.springframework.core.serializer.Deserializer;
import org.springframework.util.StreamUtils;

/**
 * A deserializer that uses an elastic {@link ByteArrayOutputStream}
 * instead of a fixed buffer. Completion is indicated by the sender
 * closing the socket.
 *
 * @author Gary Russell
 * @since 5.0
 *
 */
public class ByteArrayElasticRawDeserializer implements Deserializer<byte[]> {

    @Override
    public byte[] deserialize(InputStream inputStream) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        StreamUtils.copy(inputStream, out);
        out.close();
        return out.toByteArray();
    }

}
Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • So isn't it possible to initialize the `ByteArrayRawSerializer` with unlimited message size? Actually I don't know the message size before. – membersound Jun 02 '17 at 13:08
  • The Java language will only allow a buffer of `Integer.MAX_VALUE` - and even then only if you have enough memory. The JVM imposes further limits `Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit` – Gary Russell Jun 02 '17 at 13:12
  • The raw deserializer is simple; you would need a more sophisticated one to allocate memory as-needed (as more bytes arrive). – Gary Russell Jun 02 '17 at 13:21
  • 1
    I updated my answer with a simple elastic deserializer where the buffer will grow on demand. – Gary Russell Jun 02 '17 at 16:37
  • Fixed `StreamUtils` import. – Gary Russell Jun 02 '17 at 20:19
  • Tested today and works great. This could probably be further improved using try-with-resource on the `ByteArrayOutputStream`. – membersound Jun 06 '17 at 07:06
  • Actually, `close()` does nothing on a `ByteArrayOutputStream ` - it's not really a "resource" per se. I added the `close()` out of habit. – Gary Russell Jun 06 '17 at 12:59