How to upload a file of 750MB in JAVA, using jersey-client V2.13 (from a client to a server)?

Question

Just before starting to explain you my problem, I would like to share with you the libraries' versions that I use and the server:

javax.ws.rs-api: 2.0.1
jersey-container-servlet: 2.13
jersey-media-multipart: 2.13
jackson: 2.4.3  

I also use Apache Tomcat Server version 7.0.55. 

So I coded that below:

    /**
     * On the client side.
     */
public void uploadAFile() {
    Client client = ClientBuilder.newBuilder()
            .register(MultiPartFeature.class)
            .build();

    WebTarget target = null;
    try {
        target = client
                .target("https://blo-bla.rhcloud.com/rest")
                .path("v1").path("upload");
    } catch (IllegalArgumentException | NullPointerException e) {
        LOG_TO_CONSOLE.fatal(e, e);
        LOG_TO_FILE.fatal(e, e);
    }

    Builder builder = target.request(MediaType.TEXT_PLAIN);
    builder.header("Authorization",
    getValidBasicAuthenticationStrEncrypted());

    FormDataMultiPart form = new FormDataMultiPart();
    form.field("anotherParam", "Bozo");

    String fileName = "/Users/drizzy/Documents/Divx/CaseDepartFolder/sample.avi";

    File file = new File(fileName);
    form.bodyPart(new FileDataBodyPart("file", file,
            MediaType.APPLICATION_OCTET_STREAM_TYPE));

    Response response = builder.post(Entity.entity(form,
            MediaType.MULTIPART_FORM_DATA_TYPE));

    LOG_TO_CONSOLE.debug(response.getStatus());
    LOG_TO_CONSOLE.debug(response.readEntity(String.class));
}


   /**
     * On the server side.
     */
    @POST
    @Path("/upload")
    @Consumes(MediaType.MULTIPART_FORM_DATA)
    @Produces(MediaType.TEXT_PLAIN)
    public String uploadFile(@FormDataParam("file") InputStream fileInputStream,
                             @FormDataParam("file") FormDataContentDisposition fileDisposition,
                             @FormDataParam("anotherParam") String str) 
                             throws FileNotFoundException, IOException 
    {
        System.out.println("str: " + str);
        final String basePath = "/Users/drizzy/eclipse-workspace/tomcat7jbossews"
                                + "/src/main/resources/uploads/";

        final String fileName = fileDisposition.getFileName();
        System.out.println(new StringBuilder().append("***** fileName ")
                                              .append(fileName)
                                              .toString());

        final String filePath = new StringBuilder().append(basePath)
                                                   .append(fileName)
                                                   .toString();
        System.out.println(filePath);

        try (OutputStream fileOutputStream = new FileOutputStream(filePath)) {
            int read = 0;
            final byte[] bytes = new byte[1024];
            while ((read = fileInputStream.read(bytes)) != -1) {
                fileOutputStream.write(bytes, 0, read);
            }
        }

        return "File Upload Successfully !!";
    }

Which generates these exceptions at the client side:

Java Heap space error 

and

Java binding Exception: Already connected

So my question is to know if somebody could please provide me an example of code with a client using jersey-client V2.13 which uploads a big file from the client to the server? Or even could telling me what is wrong in my code above?

Note: I only want to use jersey-client version V2.13 for handling that problem, so please do not provide me solutions using third party libraries or which do not use jersey-client version V2.13.

Answer 1

Finally I have found a solution to my problem:

1 - For fixing the "Already connected" Exception.

Java 7 introduced SNI support which is enabled by default. I have found out that certain misconfigured servers send an "Unrecognized Name" warning in the SSL handshake which is ignored by most clients... except for Java. As @Bob Kerns mentioned, the Oracle engineers refuse to "fix" this bug/feature.

As workaround, they suggest to set the jsse.enableSNIExtension property. To allow your programs to work without re-compiling, run your app as:

"-Djsse.enableSNIExtension=false"

Source of this information here.

The property can also be set in the Java code, but it must be set before any SSL actions. Once the SSL library has loaded, you can change the property, but it won't have any effect on the SNI status. To disable SNI on runtime (with the aforementioned limitations), use:

System.setProperty("jsse.enableSNIExtension", "false");

2 - For fixing the chunkedTransferMode's activation and java heap space error.

Java heap space error: I had to increase the jvm memory used by the client:

For doing that In eclipse, go to Run > Run Configurations ... > Select the client class > Arguments > VM arguments:

-Xms1024m -Xmx2500m -XX:+PrintFlagsFinal

Sources that I used myself for fixing that problem: source 1, source 2, source 3.

Regarding the fact of enabling the chunkedTransferMode, I did not need to do that, because I found in the documentation of jersey-client that the chunkedTransferMode was enabled by default in jersey-client V2.13.

In the end we pass these parameters to the JVM at the client side:

"-Djsse.enableSNIExtension=false" "your classes ..." -Xms1024m -Xmx2500m -XX:+PrintFlagsFinal

Answer 2

Ok it seems that the problem is a really large file (350Mb) and you are hitting various limits. Therefore I would suggest to address the problem in a different way.

Possible solutions:

1. Split the file on the client and upload it via "smaller" parts to the server where it is assembled back in original order.
2. Stream a file to the server: http://commons.apache.org/proper/commons-fileupload/streaming.html

I would prefer the 2. solution as it is probably the easiest to implement. Hope this helps.