Send files with HTTP

Question

The company where I work send files using differents protocol but since few times we have a problem using HTTP protocol to a client (it's working in my VM).

@Override
    public void send(final String nf, final InputStream is)
            throws Exception {

        HttpClient http = HttpClientBuilder.create().build();

        InputStreamEntity ise = new InputStreamEntity(is);
        ise.setContentType(MediaType.APPLICATION_XML);
        ise.setChunked(true);

        HttpPost post = new HttpPost(this.serveur);
        post.setEntity(ise);

        HttpResponse r = http.execute(post);

        if (r != null) {
            int statut = r.getStatusLine().getStatusCode();
            logguer(String.format(
                    statut == HttpStatus.SC_OK ? C_LOG_TRANSMISSION_HTTP_SUCCES : C_LOG_TRANSMISSION_HTTP_FAIL,
                    statut));
        }

    }

And then we call this method into a try catch. Full log stack:

java.lang.Exception: Fail when sending.
    at com.pharmagest.dexter.plugins.transmissions.ATransmission.envoyer(ATransmission.java:262) ~[dexter-moteur.jar:na]
    at com.pharmagest.dexter.plugins.transmissions.ATransmission.transmettre(ATransmission.java:284) ~[dexter-moteur.jar:na]
    at com.pharmagest.dexter.moteur.ExecutionJob.executerTransmission(ExecutionJob.java:335) ~[dexter-moteur.jar:na]
    at com.pharmagest.dexter.moteur.ExecutionJob$$FastClassBySpringCGLIB$$9ce8f3e4.invoke(<generated>) ~[spring-core-4.0.4.RELEASE.jar:na]
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.0.4.RELEASE.jar:4.0.4.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:708) ~[spring-aop-4.0.4.RELEASE.jar:4.0.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.4.RELEASE.jar:4.0.4.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:98) ~[spring-tx-4.0.4.RELEASE.jar:4.0.4.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:262) ~[spring-tx-4.0.4.RELEASE.jar:4.0.4.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:95) ~[spring-tx-4.0.4.RELEASE.jar:4.0.4.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.4.RELEASE.jar:4.0.4.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:644) ~[spring-aop-4.0.4.RELEASE.jar:4.0.4.RELEASE]
    at com.pharmagest.dexter.moteur.ExecutionJob$$EnhancerBySpringCGLIB$$404cc5b0.executerTransmission(<generated>) ~[spring-core-4.0.4.RELEASE.jar:na]
    at com.pharmagest.dexter.ordonnanceur.JobExtraction.execute(JobExtraction.java:56) ~[dexter-ordonnanceur.jar:na]
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202) ~[quartz-2.2.1.jar:na]
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) ~[quartz-2.2.1.jar:na]
Caused by: org.apache.http.client.ClientProtocolException: null
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.3.jar:4.5.3]
    at com.pharmagest.dexter.plugins.transmissions.TransmissionHTTP.envoyer(TransmissionHTTP.java:83) ~[dexter-moteur.jar:na]
    at com.pharmagest.dexter.plugins.transmissions.ATransmission.envoyer(ATransmission.java:255) ~[dexter-moteur.jar:na]
    ... 15 common frames omitted
Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:108) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.3.jar:4.5.3]
    ... 20 common frames omitted
Caused by: java.net.SocketException: Broken pipe
    at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.7.0_25]
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109) ~[na:1.7.0_25]
    at java.net.SocketOutputStream.write(SocketOutputStream.java:153) ~[na:1.7.0_25]
    at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:377) ~[na:1.7.0_25]
    at sun.security.ssl.OutputRecord.write(OutputRecord.java:363) ~[na:1.7.0_25]
    at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:830) ~[na:1.7.0_25]
    at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:801) ~[na:1.7.0_25]
    at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122) ~[na:1.7.0_25]
    at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124) ~[httpcore-4.4.6.jar:4.4.6]
    at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136) ~[httpcore-4.4.6.jar:4.4.6]
    at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167) ~[httpcore-4.4.6.jar:4.4.6]
    at org.apache.http.impl.io.ChunkedOutputStream.flushCacheWithAppend(ChunkedOutputStream.java:122) ~[httpcore-4.4.6.jar:4.4.6]
    at org.apache.http.impl.io.ChunkedOutputStream.write(ChunkedOutputStream.java:179) ~[httpcore-4.4.6.jar:4.4.6]
    at org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:134) ~[httpcore-4.4.6.jar:4.4.6]
    at org.apache.http.impl.execchain.RequestEntityProxy.writeTo(RequestEntityProxy.java:121) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156) ~[httpcore-4.4.6.jar:4.4.6]
    at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:160) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238) ~[httpcore-4.4.6.jar:4.4.6]
    at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) ~[httpcore-4.4.6.jar:4.4.6]
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.3.jar:4.5.3]
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.3.jar:4.5.3]
    ... 22 common frames omitted

The first "caused by" error is refering to:

HttpResponse r = http.execute(post);

We already followed advices with similar errors (we add memory for example).

EDIT: As suggested @Toaster, here is my try with OkHttp3:

    OkHttpClient client = new OkHttpClient();
    MediaType xml = MediaType.parse("application/xml");

    RequestBody requestBody = RequestBodyUtil.create(xml, is);

    Request request = new Request.Builder()
            .url(this.serveur)
            .post(requestBody)
            .addHeader("Content-Type", "application/xml")
            .build();

    logguer(C_LOG_TRANSMISSION_SEPARATEUR);
    logguer(String.format(C_LOG_TRANSMISSION_HTTP_ENVOI_FICHIER, nf));

    Response response = client.newCall(request).execute(); // java.net.ProtocolException: expected 0 bytes but received 8192

    if (response != null) {
        if (response.isSuccessful()) {
            logguer(String.format(C_LOG_TRANSMISSION_HTTP_SUCCES));
        } else {
            throw new IOException("Erreur lors de la transmission HTTP " + response);
        }

    }

public class RequestBodyUtil {

    public static RequestBody create(final MediaType mediaType, final InputStream inputStream) {
        return new RequestBody() {
            @Override
            public MediaType contentType() {
                return mediaType;
            }

            @Override
            public long contentLength() {
                try {
                    return inputStream.available();
                } catch (IOException e) {
                    return 0;
                }
            }

            @Override
            public void writeTo(final BufferedSink sink)
                    throws IOException {
                Source source = null;
                try {
                    source = Okio.source(inputStream);
                    sink.writeAll(source);
                } finally {
                    Util.closeQuietly(source);
                }
            }
        };
    }
}

Trying to solve this error now:

java.net.ProtocolException: expected 0 bytes but received 8192

You use a library. It will reduce headache. You can use okhttp — exploitr, Jul 12 '18 at 09:09
Trying to do something which is already done by someone will always cause a headache — exploitr, Jul 12 '18 at 09:18
@Toaster what's the difference between using Apache HttpClient or Square OkHttp? — kumesana, Jul 12 '18 at 09:34
Most probably, Apache Httpclient supports HTTP1.1 whereas, OkHttp supports HTTP/2 OkHttp is easy to use, supports cool and fluent apis which also look nice in IDE :-) — exploitr, Jul 12 '18 at 09:41
This issue is usually caused by the peer closing the connection while you are still writing to it. For example, you have violated an upload size policy. The actual issue is to be found in the peer logs. Not in fiddling around with client code. @T — user207421, Jul 12 '18 at 10:12
Here is another library that has HttpClient util that is easy to use. Its called MgntUtils library (just make sure that you take the version 1.5.0.2 (latest at the moment) It is only in this version that HttpClient was added. Look for HttpClient description in its javadoc. The library is available at maven central and github: https://search.maven.org/#artifactdetails%7Ccom.github.michaelgantman%7CMgntUtils%7C1.5.0.2%7Cjar https://github.com/michaelgantman/Mgnt — Michael Gantman, Jul 12 '18 at 12:02
@EJP I already saw your answer in similar topic. Problem is this code works for hundred clients and for this one not that's why we are a bit confused. We send a lot of data in big files (for this one in xml) and it crashed right in the middle (maybe file too big, but already improve code for that to send few medium files and not only one very big). — Emeric, Jul 12 '18 at 12:37
@Michael Gantman Are there benefits to use this one and not OkHttp ? — Emeric, Jul 12 '18 at 12:38
@Curse - I don't have any experience with OkHttp library. So I have no idea on how MgntUtils compares to OkHttp. The reason for my comment is to give you another option so you can choose from wider range of options. It may very well be that OkHttp should be your answer. What I can tell you is that MgntUtils HttpClient is very simple and easy to use. So if you wish you can try it. — Michael Gantman, Jul 12 '18 at 14:38
@Curse code looks okay. If it worked then enjoy! and no more words. But, is that `RequestBodyUtil` class exist in official API? I checked the docs but didn't find . Instead, found `RequestBody.create(.....);` → https://square.github.io/okhttp/3.x/okhttp/okhttp3/RequestBody.html#create-okhttp3.MediaType-java.io.File- — exploitr, Jul 13 '18 at 13:10
RequestBodyUtil is a class I created (saw that in another SO), I edited. It's not working I'm getting this error: java.net.ProtocolException: expected 0 bytes but received 8192 — Emeric, Jul 13 '18 at 14:14

Send files with HTTP

0 Answers0