Java Apache http library file upload issue

Question

I Have a webservice that uploads a file. (details in the first link)

To communicate with this webservice I use org.apache.http library (Apache HttpComponents). I have found most of this code if not all of it here.

Sadly the solution only works with images and despite the efforts when trying to upload a video is shows the error of Content Length too long. To try and fix this I decided to replace it with what's in current use. At first using HttpClientBuilder

HttpClient httpclient = HttpClientBuilder.create().disableContentCompression().build();

My try failed as it still gives the same error and after research I found out that I need to use the CloseableHttpClient as follow to disable the automatic headers

CloseableHttpClient httpclient = HttpClients.custom()
        .setHttpProcessor(HttpProcessorBuilder.create().build())
        .build();

That worked but when setting the headers something doesn't pass I can't figure it out. I did it as follow

httppost.addHeader("Keep-Alive", "timeout=5, max=100");
httppost.addHeader("Connection", "Keep-Alive");
httppost.addHeader("Content-Type", "text/html; charset=UTF-8");

But when tracing the response this is what I have

Http Client Builder Trace
    Date,Tue, 28 Mar 2017 18:30:50 GMT
    Server,Apache/2.4.23 (Win64) PHP/7.0.10
    X-Powered-By,PHP/7.0.10
    Content-Length,43
    Keep-Alive,timeout=5, max=100
    Connection,Keep-Alive
    Content-Type,text/html; charset=UTF-8


Closable Response Trace
    Date,Tue, 28 Mar 2017 18:31:27 GMT
    Server,Apache/2.4.23 (Win64) PHP/7.0.10
    Content-Length,311
    Connection,close
    Content-Type,text/html; charset=iso-8859-1

I tried to set the headers to the response but I am haven't figured how to instantiate it properly to set the headers. And also I am trying to understand why the httppost is not being taken in consideration. Did I miss something?

One last thing when I use Closable The way I did it doesn't upload anymore due to missing headers I guess.

EDIT

Error when uploading video

Exception in thread "main" org.apache.http.ContentTooLongException: Content length is too long: 16363535
at org.apache.http.entity.mime.MultipartFormEntity.getContent(MultipartFormEntity.java:103)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:199)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:306)
at testfileupload.PostFile.main(PostFile.java:100)
C:\Users\User\AppData\Local\NetBeans\Cache\8.2\executor-snippets\run.xml:53: Java returned: 1
BUILD FAILED (total time: 0 seconds)

EDIT

I changed the max file size for php following this but it still shows the same error

Update

This is how I handle the multipart (I use multipartform)

HttpEntity resEntity = MultipartEntityBuilder.create()
    .addBinaryBody("userfile", file, ContentType.create("video/mp4"), file.getName())
    .build();

I rememeber reading something about it (not sure) And as for the php.ini params that need to be of higher value upload_max_filesize = 40M and post_max_size = 40M and memory_limit = 128M which I am kinda sure its enough (using wamp64).

I kept trying to udnerstand the content length issue since ..

I found out that the HttpEntity sets calculates the length and I checked the value its 2 bits off I believe but no matter the conenction type it just doesn't work and sticks to content length error.

I also tried

HttpClient httpclient = HttpClients.custom().setDefaultHeaders(Arrays.asList(CustomHeader)).build();

But still nothing. At this point I highly doubt its about setting the header and more related to HttpEntity with the Multipart or the HttpPost but It could be else.

Upload

I tried with a 9s Video of a size ~350Kb but it still gives the content length error which means there might be a default content set or none at all

When you go the original content length error, was it a problem with the library like you seemed to indicate or a limitation on the file size for the upload in php. PHP requires configuration via php.ini for the sizes on the server. The default is smaller than a video file. — Lucas Holt, Mar 28 '17 at 19:22
If my math is right, that's a 16MB file give or take. Many versions of php limit to 2MB by default. Can you check what php.ini is set to on the server? These three parameters are limits in php.ini memory_limit = 32M upload_max_filesize = 24M post_max_size = 32M If these settings are too low, no matter what you change in the client, it will not work. — Lucas Holt, Mar 28 '17 at 22:30
Also, you're going to want to change the content type header to match the file you are uploading. It's certainly not html. See http://stackoverflow.com/questions/51438/getting-a-files-mime-type-in-java — Lucas Holt, Mar 28 '17 at 22:31

score 0 · Answer 1 · edited May 23 '17 at 10:30

So I dont know what I did / changed but now its working with the error.

I would really want to know why this cause an error it doesn't make sense so far unless its calling something that can't take too high of a value (method of param type given value more than it can)

To be clear the error is generated when I use this (verifcation area)

System.out.println(response.getStatusLine());
if (resEntity != null) {
// error from below, kinda forgot about it
      System.out.println(EntityUtils.toString(resEntity));
}

    if (resEntity != null) {
//      resEntity.consumeContent();
    }

The webservice is the same as in the first link.

Below is my code (dont mind the comments I didn't clear the decrepated / changed / replaced / test / different approaches)

import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.apache.http.Header;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpResponse;
import org.apache.http.HttpVersion;
import org.apache.http.ProtocolVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.mime.MultipartEntity;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.ContentBody;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.params.CoreProtocolPNames;
import org.apache.http.protocol.HTTP;
import org.apache.http.protocol.HttpProcessorBuilder;
import org.apache.http.util.EntityUtils;


public class PostFile {
  public static void main(String[] args) throws Exception {
//    HttpClient httpclient = new DefaultHttpClient();

//    CloseableHttpClient httpclient = HttpClients.custom()
//        .setHttpProcessor(HttpProcessorBuilder.create().build())
//        .build();

//    Header headerCustom = new BasicHeader(HttpHeaders.CONTENT_LENGTH, "15");


//    HttpClient httpclient = HttpClients.custom().build();

//        HttpClient httpclient = HttpClientBuilder.create().disableContentCompression().build();
   HttpClient httpclient = HttpClientBuilder.create().build();     
//    httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);


    HttpPost httppost = new HttpPost("http://localhost/userVids/upload.php");

    httppost.setProtocolVersion(new ProtocolVersion("HTTP", 1, 1));
    File file = new File("C:/Users/User/Desktop/test_video.mp4");

//    httppost.addHeader("Content-Length", String.valueOf(file.length()));

//    MultipartEntity mpEntity = new MultipartEntity();
//    ContentBody cbFile = new FileBody(file, ContentType.MULTIPART_FORM_DATA);
//    mpEntity.addPart("userfile", cbFile);
//      System.out.println(mpEntity.getContentLength());
//
//    httppost.setEntity(mpEntity);

    /*
        Date,Tue, 28 Mar 2017 18:30:50 GMT
        Server,Apache/2.4.23 (Win64) PHP/7.0.10
        X-Powered-By,PHP/7.0.10
        Content-Length,43
        Keep-Alive,timeout=5, max=100
        Connection,Keep-Alive
        Content-Type,text/html; charset=UTF-8
    */

    /*
        Date,Tue, 28 Mar 2017 18:31:27 GMT
        Server,Apache/2.4.23 (Win64) PHP/7.0.10
        Content-Length,311
        Connection,close
        Content-Type,text/html; charset=iso-8859-1
    */

    HttpEntity resEntity = MultipartEntityBuilder.create()
        .addBinaryBody("userfile", file, ContentType.create("video/mp4"), file.getName())
        .build();

      System.out.println("Entity Length " + resEntity.getContentLength());
//    httppost.addHeader("Keep-Alive", "timeout=5, max=100");
//    httppost.addHeader("Connection", "Keep-Alive");
//    httppost.addHeader("Content-Type", "text/html; charset=UTF-8");

//    httppost.addHeader("Content-Disposition", "form-data; name=\"userfile\"; filename=\"star_empty.png\"");
//    httppost.addHeader("Content-Type", "image/png");
//    httppost.addHeader("Content-Transfer-Encoding", "binary");
//    httppost.addHeader("Content-Length", "99999");

    httppost.setEntity(resEntity);

    System.out.println("executing request " + httppost.getRequestLine());
    HttpResponse response = httpclient.execute(httppost);
//    HttpEntity resEntity = response.getEntity();

    System.out.println("Headers (name,value)");
    List<Header> httpHeaders = Arrays.asList(response.getAllHeaders());        
    httpHeaders.forEach((header) -> {
        System.out.println(header.getName() + "," + header.getValue());
    });

    System.out.println(response.getStatusLine());
    if (resEntity != null) {
      System.out.println(EntityUtils.toString(resEntity));
    }
    if (resEntity != null) {
//      resEntity.consumeContent();
    }

//    httpclient.getConnectionManager().shutdown();
  }
}

BUT A HUGE NOTICE I had my php.ini (changed all of them in wamp didn't know if it would consider only the running version or reads all (didn't test) ) and changed the values as Lucas Holt (he fixed it i.e, I am just providing for who needs or maybe explanation) said and he deserves credit for it. (I am sure it didn't work before that I just tested a lot and dont remember really.)

Java Apache http library file upload issue

1 Answers1