1

So I developed a spring boot microsservice, that accepts a multipart file and then forwards it to another service that is running outside of kubernetes network.
When I upload file larger than 20mb I get the following error: 

java.lang.OutOfMemoryError: Java heap space
    at java.base/java.util.Arrays.copyOf(Unknown Source) ~[na:na]
    at java.base/java.io.ByteArrayOutputStream.grow(Unknown Source) ~[na:na]
    at java.base/java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source) ~[na:na]
    at java.base/java.io.ByteArrayOutputStream.write(Unknown Source) ~[na:na]
    at org.springframework.util.StreamUtils.copy(StreamUtils.java:143) ~[spring-core-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.http.converter.ResourceHttpMessageConverter.writeContent(ResourceHttpMessageConverter.java:132) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.http.converter.ResourceHttpMessageConverter.writeInternal(ResourceHttpMessageConverter.java:124) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.http.converter.ResourceHttpMessageConverter.writeInternal(ResourceHttpMessageConverter.java:45) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.http.converter.AbstractHttpMessageConverter.write(AbstractHttpMessageConverter.java:227) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.http.converter.FormHttpMessageConverter.writePart(FormHttpMessageConverter.java:417) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.http.converter.FormHttpMessageConverter.writeParts(FormHttpMessageConverter.java:393) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.http.converter.FormHttpMessageConverter.writeMultipart(FormHttpMessageConverter.java:373) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:277) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.http.converter.FormHttpMessageConverter.write(FormHttpMessageConverter.java:95) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]
    at org.springframework.web.client.RestTemplate$HttpEntityRequestCallback.doWithRequest(RestTemplate.java:948) ~[spring-web-5.1.9.RELEASE.jar!/:5.1.9.RELEASE]

The service itself doesnt crash. The amount of ram shouldn't be an issue. At first I set it up to be 512MB and then increased it to 1024MB 

   limits:
     cpu: 200m
     memory: 1024Mi
   requests:
     cpu: 200m
     memory: 1024Mi

The java code handling the upload:

params.add("file", new MultipartInputStreamFileResource(multipartfile.getInputStream(),
                    multipartfile.getOriginalFilename()));

            UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(requestUrl);
            for (Entry<String, List<Object>> entry : params.entrySet()) {
                builder.queryParam(entry.getKey(), entry.getValue());
            }

            HttpEntity<LinkedMultiValueMap<String, Object>> requestEntity = new HttpEntity<>(params, headers);
            restTemplate.postForObject(requestUrl, requestEntity, String.class);

Java spring is running at default settings. I only increased tomcat's max upload file...

Kristjan
  • 409
  • 5
  • 18
  • I assume this is memory configuration of the Kubernetes container. What parameters are you using when running JVM? – syntagma Feb 06 '20 at 17:13
  • How do I check this? I think I use default ones – Kristjan Feb 06 '20 at 17:54
  • Did you try running it without limits? 1GB might not be enough. Additionally try setting up xmx- https://stackoverflow.com/a/14763095/12237732 – kool Feb 07 '20 at 09:42
  • I tried running it without limits. but this is not the right way... according to my coworkers. 512mb should be enough to handle this with streaming. – Kristjan Feb 10 '20 at 09:48
  • Try increasing the max heap size (default is 64Mb). It can be passed as env variable. – kool Feb 10 '20 at 10:52
  • Is this with only 1 request or multiple requests? 200m CPU hard limits is probably too little. You could be throttling your CPU which increases your processing time. And if you get a few requests queued up, there goes the memory. Does this service connect to a DB? What other dependencies does it have? Can you post your Dockerfile as well? Thanks – Urosh T. Feb 12 '20 at 20:03

0 Answers0