23

I've got the following code for a file upload with Apache's HTTP-Client (org.apache.http.client):

  public static void main(String[] args) throws Exception
  {
    String fileName = "test.avi";
    File file = new File(fileName);

    String serverResponse = null;
    HttpParams params = new BasicHttpParams();
    params.setParameter(HttpProtocolParams.USE_EXPECT_CONTINUE, true);
    HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
    HttpClient client = new DefaultHttpClient(params);
    HttpPut put = new HttpPut("http://localhost:8080/" + fileName);

    FileEntity fileEntity = new FileEntity(file, "binary/octet-stream");
    put.setEntity(fileEntity);   

    HttpResponse response = client.execute(put);
    HttpEntity entity = response.getEntity();
    if (entity != null)
    {
      serverResponse = EntityUtils.toString(entity);
      System.out.println(serverResponse);
    }
  }

It work's quite well but now I want to have a progress bar which shows the progress of the file upload. How can this be made? I found a code snippet at File Upload with Java (with progress bar) but it is designed for Apache HTTP Client 3 (org.apache.commons.httpclient) and the RequestEntity class does not exist in Apache HTTP Client 4. ;(

Maybe someone of you has an approach?

Many greetings

Benny

Community
  • 1
  • 1
Benny Code
  • 51,456
  • 28
  • 233
  • 198
  • is there a way you can get the ammount of bytes uploaded? plz add a code snippet of that and i may have your answer – Stas Jaro Aug 14 '11 at 15:26
  • Thank you for your helpfulness! I have found a solution myself and will introduce it in a timely manner. :) – Benny Code Aug 14 '11 at 23:41
  • You may also like: http://stackoverflow.com/questions/22932821/httpclient-post-with-progress-and-multipartentitybuilder/22942382#22942382 it features a wrapping solution which is less intrusive to your class hierarchy. – yankee May 01 '14 at 23:47

4 Answers4

21

I introduced a derived FileEntity that just counts the written bytes. It uses OutputStreamProgress that does the actual counting (kind of a decorator to the actual OutputStream).

The advantage of this (and decoration in general) is that I do not need to copy the actual implementation, like the the actual copying from the file stream to the output stream. I can also change to use a different (newer) implementation, like the NFileEntity.

Enjoy...

FileEntity.java

public class FileEntity extends org.apache.http.entity.FileEntity {

    private OutputStreamProgress outstream;

    public FileEntity(File file, String contentType) {
        super(file, contentType);
    }

    @Override
    public void writeTo(OutputStream outstream) throws IOException {
        this.outstream = new OutputStreamProgress(outstream);
        super.writeTo(this.outstream);
    }

    /**
     * Progress: 0-100
     */
    public int getProgress() {
        if (outstream == null) {
            return 0;
        }
        long contentLength = getContentLength();
        if (contentLength <= 0) { // Prevent division by zero and negative values
            return 0;
        }
        long writtenLength = outstream.getWrittenLength();
        return (int) (100*writtenLength/contentLength);
    }
}

OutputStreamProgress.java

public class OutputStreamProgress extends OutputStream {

    private final OutputStream outstream;
    private volatile long bytesWritten=0;

    public OutputStreamProgress(OutputStream outstream) {
        this.outstream = outstream;
    }

    @Override
    public void write(int b) throws IOException {
        outstream.write(b);
        bytesWritten++;
    }

    @Override
    public void write(byte[] b) throws IOException {
        outstream.write(b);
        bytesWritten += b.length;
    }

    @Override
    public void write(byte[] b, int off, int len) throws IOException {
        outstream.write(b, off, len);
        bytesWritten += len;
    }

    @Override
    public void flush() throws IOException {
        outstream.flush();
    }

    @Override
    public void close() throws IOException {
        outstream.close();
    }

    public long getWrittenLength() {
        return bytesWritten;
    }
}
AlikElzin-kilaka
  • 34,335
  • 35
  • 194
  • 277
  • 1
    Love the decorator idea. I extended it some in another answer. – Jazzepi Aug 09 '12 at 17:14
  • Where did you extend it? Perhaps it may be of value here. – AlikElzin-kilaka Aug 09 '12 at 22:31
  • Woops! I meant to vote it up. I responded to this thread just below yours. – Jazzepi Aug 10 '12 at 01:52
  • 4
    Isn't this just monitoring the progress of the file being written to the output stream rather than the progress of the actual upload? If I understand correctly, the actual upload doesn't occur until the output stream is flushed. – Brian Nov 15 '13 at 17:01
9

A new version using the package org.apache.commons.io.output from commons-io (2.4) and its class CountingOutputStream.

I changed the initial code to reflect my project needs to use a multipart form as input and the post method (this dues to the requirements imposed by the server side).

Consider that the delta of large file correspond in my tests to 4096 bytes. This means that the listener method counterChanged() is called every 4096 bytes of transfered data, what is acceptable for my use case.

The method looks like:

public void post(String url, File sendFile) {
    HttpParams params = new BasicHttpParams();
    params.setParameter(HttpProtocolParams.USE_EXPECT_CONTINUE, true);
    HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
    HttpClient client = new DefaultHttpClient(params);
    HttpPost post = new HttpPost(url + "/" + sendFile.getName());
    MultipartEntity multiEntity = new MultipartEntity(); 
    MyFileBody fileBody = new MyFileBody(sendFile);

    fileBody.setListener(new IStreamListener(){

        @Override
        public void counterChanged(int delta) {
            // do something
            System.out.println(delta);
        }});

    multiEntity.addPart("file", fileBody);
    StringBody stringBody = new StringBody(sendFile.getName());
    multiEntity.addPart("fileName", stringBody);
    post.setEntity(multiEntity);   
    HttpResponse response = client.execute(post);
}

The class MyFileBody becomes:

public class MyFileBody extends FileBody {

    private IStreamListener listener;

    public MyFileBody(File file) {
        super(file);
    }

    @Override
    public void writeTo(OutputStream out) throws IOException {
        CountingOutputStream output = new CountingOutputStream(out) {
            @Override
            protected void beforeWrite(int n) {
                if (listener != null && n != 0)
                    listener.counterChanged(n);
                super.beforeWrite(n);
            }
        };
        super.writeTo(output);

    }

    public void setListener(IStreamListener listener) {
        this.listener = listener;
    }

    public IStreamListener getListener() {
        return listener;
    }

}

Finally, the listener interface looks like:

public interface IStreamListener {

    void counterChanged(int delta);

}
Marco Nanni
  • 113
  • 2
  • 6
7

This answer extends kilaka's answer by adding a simple listener to the OutputStreamProgress.java class instead of having the public getProgress() method (I'm honestly not sure how you are suppose to call the getProgress() method since the thread will be executing inside of httpclient's code the entire time you might want to call getProgress()!).

Please note you'll need to extend the entity class for each entity type you want to use, and when you write your HttpClient code, you'll need to create the entity of that new type.

I wrote a very basic write listener that implements the WriteListener interface. This is where you'll add your logic to do something with the write reports from the OutputStreamProgress, something like updating a progress bar :)

Big thanks to kilaka for using the decorator idea to sneak in a counting outstream.

WriteLisener.java

public interface WriteListener {
    void registerWrite(long amountOfBytesWritten);
}

OutputStreamProgress.java

import java.io.IOException;
import java.io.OutputStream;

public class OutputStreamProgress extends OutputStream {

    private final OutputStream outstream;
    private long bytesWritten=0;
    private final WriteListener writeListener;
    public OutputStreamProgress(OutputStream outstream, WriteListener writeListener) {
        this.outstream = outstream;
        this.writeListener = writeListener;
    }

    @Override
    public void write(int b) throws IOException {
        outstream.write(b);
        bytesWritten++;
        writeListener.registerWrite(bytesWritten);
    }

    @Override
    public void write(byte[] b) throws IOException {
        outstream.write(b);
        bytesWritten += b.length;
        writeListener.registerWrite(bytesWritten);
    }

    @Override
    public void write(byte[] b, int off, int len) throws IOException {
        outstream.write(b, off, len);
        bytesWritten += len;
        writeListener.registerWrite(bytesWritten);
    }

    @Override
    public void flush() throws IOException {
        outstream.flush();
    }

    @Override
    public void close() throws IOException {
        outstream.close();
    }
}

BasicWriteListener

public class BasicWriteListener implements WriteListener {

public BasicWriteListener() {
    // TODO Auto-generated constructor stub
}

public void registerWrite(long amountOfBytesWritten) {
    System.out.println(amountOfBytesWritten);
}

}

MultipartEntityWithProgressBar

import java.io.IOException;
import java.io.OutputStream;
import java.nio.charset.Charset;

import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntity;

public class MultipartEntityWithProgressBar extends MultipartEntity {
    private OutputStreamProgress outstream;
    private WriteListener writeListener;

    @Override
    public void writeTo(OutputStream outstream) throws IOException {
        this.outstream = new OutputStreamProgress(outstream, writeListener);
        super.writeTo(this.outstream);
    }

    public MultipartEntityWithProgressBar(WriteListener writeListener)
    {
        super();
        this.writeListener = writeListener;
    }
    public MultipartEntityWithProgressBar(HttpMultipartMode mode, WriteListener writeListener)
    {
        super(mode);
        this.writeListener = writeListener;
    }
    public MultipartEntityWithProgressBar(HttpMultipartMode mode, String boundary, Charset charset, WriteListener writeListener)
    {
        super(mode, boundary, charset);
        this.writeListener = writeListener;
    }

    // Left in for clarity to show where I took from kilaka's answer
//  /**
//   * Progress: 0-100
//   */
//  public int getProgress() {
//      if (outstream == null) {
//          return 0;
//      }
//      long contentLength = getContentLength();
//      if (contentLength <= 0) { // Prevent division by zero and negative values
//          return 0;
//      }
//      long writtenLength = outstream.getWrittenLength();
//      return (int) (100*writtenLength/contentLength);
//  }

}
Jazzepi
  • 5,259
  • 11
  • 55
  • 81
  • In the OutputStreamProgress, could you explain why the bytesWritten variable is declared as volatile? This [article](http://www.javamex.com/tutorials/synchronization_volatile_dangers.shtml) says that it may not be thread-safe when used with '++' and '+=', is that a potential bug in this situation? Thanks – bmeding Dec 31 '12 at 22:16
  • @bmeding I'm not sure how that volatile keyword got in there! I wrote this class a long time ago, but I've edited to remove it from my answer, and I don't believe its necessary at all. – Jazzepi Jan 01 '13 at 05:51
5

Hello guys!

I solved the problem myself and made ​​a simple example to it.
If there are any questions, feel free to ask.

Here we go!

ApplicationView.java

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.io.File;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JProgressBar;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.HttpVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPut;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.HttpParams;
import org.apache.http.params.HttpProtocolParams;
import org.apache.http.util.EntityUtils;

public class ApplicationView implements ActionListener
{

  File file = new File("C:/Temp/my-upload.avi");
  JProgressBar progressBar = null;

  public ApplicationView()
  {
    super();
  }

  public void createView()
  {
    JFrame frame = new JFrame("File Upload with progress bar - Example");
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    frame.setBounds(0, 0, 300, 200);
    frame.setVisible(true);

    progressBar = new JProgressBar(0, 100);
    progressBar.setBounds(20, 20, 200, 30);
    progressBar.setStringPainted(true);
    progressBar.setVisible(true);

    JButton button = new JButton("upload");
    button.setBounds(progressBar.getX(),
            progressBar.getY() + progressBar.getHeight() + 20,
            100,
            40);
    button.addActionListener(this);

    JPanel panel = (JPanel) frame.getContentPane();
    panel.setLayout(null);
    panel.add(progressBar);
    panel.add(button);
    panel.setVisible(true);
  }

  public void actionPerformed(ActionEvent e)
  {
    try
    {
      sendFile(this.file, this.progressBar);
    }
    catch (Exception ex)
    {
      System.out.println(ex.getLocalizedMessage());
    }
  }

  private void sendFile(File file, JProgressBar progressBar) throws Exception
  {
    String serverResponse = null;
    HttpParams params = new BasicHttpParams();
    params.setParameter(HttpProtocolParams.USE_EXPECT_CONTINUE, true);
    HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
    HttpClient client = new DefaultHttpClient(params);
    HttpPut put = new HttpPut("http://localhost:8080/" + file.getName());

    ProgressBarListener listener = new ProgressBarListener(progressBar);
    FileEntityWithProgressBar fileEntity = new FileEntityWithProgressBar(file, "binary/octet-stream", listener);
    put.setEntity(fileEntity);

    HttpResponse response = client.execute(put);
    HttpEntity entity = response.getEntity();
    if (entity != null)
    {
      serverResponse = EntityUtils.toString(entity);
      System.out.println(serverResponse);
    }
  }
}

FileEntityWithProgressBar.java

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.http.entity.AbstractHttpEntity;

/**
 * File entity which supports a progress bar.<br/>
 * Based on "org.apache.http.entity.FileEntity".
 * @author Benny Neugebauer (www.bennyn.de)
 */
public class FileEntityWithProgressBar extends AbstractHttpEntity implements Cloneable
{

  protected final File file;
  private final ProgressBarListener listener;
  private long transferredBytes;

  public FileEntityWithProgressBar(final File file, final String contentType, ProgressBarListener listener)
  {
    super();
    if (file == null)
    {
      throw new IllegalArgumentException("File may not be null");
    }
    this.file = file;
    this.listener = listener;
    this.transferredBytes = 0;
    setContentType(contentType);
  }

  public boolean isRepeatable()
  {
    return true;
  }

  public long getContentLength()
  {
    return this.file.length();
  }

  public InputStream getContent() throws IOException
  {
    return new FileInputStream(this.file);
  }

  public void writeTo(final OutputStream outstream) throws IOException
  {
    if (outstream == null)
    {
      throw new IllegalArgumentException("Output stream may not be null");
    }
    InputStream instream = new FileInputStream(this.file);
    try
    {
      byte[] tmp = new byte[4096];
      int l;
      while ((l = instream.read(tmp)) != -1)
      {
        outstream.write(tmp, 0, l);
        this.transferredBytes += l;
        this.listener.updateTransferred(this.transferredBytes);
      }
      outstream.flush();
    }
    finally
    {
      instream.close();
    }
  }

  public boolean isStreaming()
  {
    return false;
  }

  @Override
  public Object clone() throws CloneNotSupportedException
  {
    return super.clone();
  }
}

ProgressBarListener.java

import javax.swing.JProgressBar;

public class ProgressBarListener
{

  private int transferedMegaBytes = 0;
  private JProgressBar progressBar = null;

  public ProgressBarListener()
  {
    super();
  }

  public ProgressBarListener(JProgressBar progressBar)
  {
    this();
    this.progressBar = progressBar;
  }

  public void updateTransferred(long transferedBytes)
  {
    transferedMegaBytes = (int) (transferedBytes / 1048576);
    this.progressBar.setValue(transferedMegaBytes);
    this.progressBar.paint(progressBar.getGraphics());
    System.out.println("Transferred: " + transferedMegaBytes + " Megabytes.");
  }
}

Happy Coding!

Benny Code
  • 51,456
  • 28
  • 233
  • 198