Spring Boot Multipart File Upload Size Inconsistency

Question

I have an endpoint that uploads an image file to the server, then to S3.

When I run on localhost, the MultipartFile byte size is correct, and the upload is successful.

However, the moment I deploy it to my EC2 instance the uploaded file size is incorrect.

Controller Code

@PostMapping("/{id}/photos")
fun addPhotos(@PathVariable("id") id: Long,
              @RequestParam("file") file: MultipartFile,
              jwt: AuthenticationJsonWebToken) = ApiResponse.success(propertyBLL.addPhotos(id, file, jwt))

Within the PropertyBLL.addPhotos method, printing file.size results in the wrong size.

Actual file size is 649305 bytes, however when uploaded to my prod server it reads as 1189763 bytes.

• My production server is an AWS EC2 instance, behind Https.
• The Spring application yml files are the same. The only configurations I overrode were the file max size properties.
• I'm using PostMan to Post the request. I'm passing the body as form-data, key named "file".
• Again, it works perfectly when running locally.

I did another test where I wrote the uploaded file to the server so I could compare.

Uploaded file's first n bytes in Hex editor:

EFBFBD50 4E470D0A 1A0A0000 000D4948 44520000 03000000 02400802 000000EF BFBDCC96 01000000 0467414D 410000EF BFBDEFBF BD0BEFBF BD610500 00002063 48524D00 007A2600

Original file's first n bytes:

89504E47 0D0A1A0A 0000000D 49484452 00000300 00000240 08020000 00B5CC96 01000000 0467414D 410000B1 8F0BFC61 05000000 20634852 4D00007A 26000080 840000FA 00000080

They both appear to have the text "PNG" in them and also have the ending EXtdate:modify/create markers.

Per Request, the core contents of addPhoto:

val metadata = ObjectMetadata()
metadata.contentLength = file.size
metadata.contentType = "image/png"
LOGGER.info("Uploading image of size {} bytes, name: {}", file.size, file.originalFilename)
val request = PutObjectRequest(awsProperties.cdnS3Bucket, imageName, file.inputStream, metadata)
awsSdk.putObject(request)

This works when I run web server locally. imageName is just a custom built name. There is other code involving hibernate models, but is not relevant.

Update

This appears to be Https/api proxy related. When I hit the EC2 node's http url, it works fine. However, when I go through the api proxy (https://api.thedomain.com), which proxies to the EC2 node, it fails. I will continue down this path.

Answer 1

After more debugging I discovered that when I POST to the EC2 instance directly everything works as expected. Our primary and public api url makes proxies requests through Amazon's API Gateway service. This service for some reason converts the data to Base64 instead of just passing through raw binary data.

I have found documentation to update the API Gateway to passthrough binary data: here.

I am using the Content-Type value of multipart/form-data. Do not forget to also add it in your API Settings where you enable binary support.

I did not have to edit the headers options, additionally I used the default "Method Request Passthrough" template.

And finally, don't forget to deploy your api changes...

It's now working as expected.

Answer 2

Sorry, but many of the comments make no sense. file.size will return the size of the uploaded file in bytes, NOT the size of the request (which, yes, due to different filters could potentially be enhanced with additional information and increase in size). Spring can't just magically double the size of a PNG file (in your case adding almost another ~600kb of information on top of whatever you've sent). While I'd like to trust that you know what you're doing and the numbers you are giving us are indeed correct, to me, all evidence points to human error... please, double-, triple-, quadruple- check that you're indeed uploading the same file in all scenarios.

How did you get to 649305 bytes in the first place? Who gave you that number? Was it your code or did you actually look at the file on disk and see how big it was? The only way compression discussions make any sense in this context is if 649305 bytes is the already compressed size of the file when running locally (it's actual size on disk being 1189763 bytes) and indeed, compression not being turned on when deployed to AWS for some reason and you receive the full uncompressed file (we don't even know how you are deploying it... is it really the same as locally? Are you running a standalone .jar in both cases? Are you deploying a .war to AWS perhaps instead? Are you really running the app in the same container and container version in both cases or are you perhaps running Tomcat locally and Jetty on AWS? etc. etc. etc.). Are you sure your Postman request is not messed up and you're not sending something else by accident (or more than you think)?

EDIT:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>com.sandbox</groupId>
    <artifactId>spring-boot-file-upload</artifactId>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.6.RELEASE</version>
    </parent>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

package com.sandbox;

import static org.springframework.http.ResponseEntity.ok;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;

@SpringBootApplication
public class Application {

    public static void main(final String[] arguments) {
        SpringApplication.run(Application.class,
                              arguments);
    }

    @RestController
    class ImageRestController {

        @PostMapping(path = "/images")
        ResponseEntity<String> upload(@RequestParam(name = "file") final MultipartFile image) {
            return ok("{\"response\": \"Uploaded image having a size of '" + image.getSize() + "' byte(s).\"}");
        }
    }
}

The example is in Java because it was just faster to put together (the environment is a simple Java environment having the standalone .jar deployed - no extra configs or anything, except for the server port being on 5000). Either way, you can try it out yourself by sending POST requests to http://test123456.us-east-1.elasticbeanstalk.com/images

This is my Postman request and the response using the image you've provided:

Everything seems to be looking fine on my AWS EB instance and all the numbers add up as expected. If you are saying your setup is as simple as it sounds then I'm unfortunately just as puzzled as you are. I can only assume that there's more to what you have shared so far (however, I doubt the issue is related to Spring Boot... then it is more likely that it has to do with your AWS configs/setup).

Answer 3

CloudFormation Template snippet for achieving Kenny Cason's solution:

  MyApi:
    Type: AWS::Serverless::Api
    Properties:
      BinaryMediaTypes:
        - "multipart/form-data"