1

I am trying to get the progress of the actual file upload using HttpPost. I have a stable solution so far (that i found here in SO) that has a progress but after uploading large files i realized it only counts the bytes written to the output buffer NOT the post transfer progress. I would like to somehow get the progress of the actual "post". Can someone explain how I can achieve this using what I worked so hard to get so far? Most of the solutions I have found online are only counting bytes written to output buffer (which is good enough for small files but not when transferring large files).

public static String postFile(final Context context, String fileName) throws Exception {

    HttpClient client = new DefaultHttpClient();
    HttpPost post = new HttpPost("http://my.url/");
    MultipartEntityBuilder builder = MultipartEntityBuilder.create();        
    builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);

    final File file = new File(fileName);
    final long totalSize = file.length();
    FileBody fb = new FileBody(file);

    builder.addPart("uploaded_file", new FileBody(new File(fileName)));


    final HttpEntity yourEntity = builder.build();

    int progressPercent = 0;

    class ProgressiveEntity implements HttpEntity {
        @Override
        public void consumeContent() throws IOException {
            yourEntity.consumeContent();                
        }
        @Override
        public InputStream getContent() throws IOException,
                IllegalStateException {
            return yourEntity.getContent();
        }
        @Override
        public Header getContentEncoding() {             
            return yourEntity.getContentEncoding();
        }
        @Override
        public long getContentLength() {
            return yourEntity.getContentLength();
        }
        @Override
        public Header getContentType() {
            return yourEntity.getContentType();
        }
        @Override
        public boolean isChunked() {             
            return yourEntity.isChunked();
        }
        @Override
        public boolean isRepeatable() {
            return yourEntity.isRepeatable();
        }
        @Override
        public boolean isStreaming() {             
            return yourEntity.isStreaming();
        } // CONSIDER put a _real_ delegator into here!

        @Override
        public void writeTo(OutputStream outstream) throws IOException {

            class ProxyOutputStream extends FilterOutputStream {

                public ProxyOutputStream(OutputStream proxy) {
                    super(proxy);    
                }
                public void write(int idx) throws IOException {
                    out.write(idx);
                }
                public void write(byte[] bts) throws IOException {
                    out.write(bts);
                }
                public void write(byte[] bts, int st, int end) throws IOException {
                    out.write(bts, st, end);
                }
                public void flush() throws IOException {
                    out.flush();
                }
                public void close() throws IOException {
                    out.close();
                }
            } // CONSIDER import this class (and risk more Jar File Hell)

            class ProgressiveOutputStream extends ProxyOutputStream {
                long totalSent;
                public ProgressiveOutputStream(OutputStream proxy) {
                       super(proxy);
                       totalSent = 0;
                }

                public void write(byte[] bts, int st, int end) throws IOException {

                // end is the amount being sent this time
                // st is always zero and end=bts.length()

                     totalSent += end;
                     int progress = (int) ((totalSent / (float) totalSize) * 100);
                     out.write(bts, st, end);
                }
            }

            yourEntity.writeTo(new ProgressiveOutputStream(outstream));
        }

    };



    ProgressiveEntity myEntity = new ProgressiveEntity();

    post.setEntity(myEntity);

    //Output to buffer is complete at this point!
    HttpResponse response = client.execute(post);        

    String jsonResponseStr = getContent(response);

    Log.d("MYTAG",jsonResponseStr);

    return jsonResponseStr;

} 

In my receiving script on the remote server, I am only echo-ing a string so that I can send an immediate response (no file/database processing at all) and the response from server still takes a very long time. I strongly believe at this point the transfer happens after write to buffer completes.

el producer
  • 321
  • 3
  • 9
  • Use publishProgress(); Check [this](http://stackoverflow.com/questions/6924447/how-to-implement-file-upload-progress-bar-in-android) – MrDumb Oct 17 '14 at 05:21
  • @ashutiwari4 that answer is also counting the write to buffer progress unless you use the setfixedlengthstreamingmode option which I dont think is a method of anything im using. I would like to avoid rewriting everything I have if possible. – el producer Oct 17 '14 at 13:41
  • `NOT the post transfer progress.`. What is a post transfer progress? – greenapps Oct 17 '14 at 20:16
  • @greenapps When you write to buffer, the POST has not actually started. After write to buffer completes, the post to remote url begins. I want to track progress of the POST – el producer Oct 17 '14 at 20:37
  • So you say that i can write a 5 MB file to the output stream and when done the actual transmitting will start? Well i have difficulties to believe that. – greenapps Oct 17 '14 at 21:48
  • @greenapps Yes, that's exactly what I'm saying. It is more apparent with a 100mb file. I didn't realize it until I tried a 100mb file. WIth small files its possible to send without buffering them. With larger files you will get a memory error which is why we need to buffer the file to an output stream. – el producer Oct 18 '14 at 02:45
  • Now you are telling yet another story. I really don,'t know where you are talking about. We were not talking about buffering yes or not. You said that actual sending only started when writes were done. Which i still don't believe by the way. – greenapps Oct 18 '14 at 07:57
  • i removed all processing code on server so i can send an immediate response from server after POSTing file and still takes a very long time to get a response after seding 130mb file. I have to believe transfer happens after buffer reaches 100%. Please prove me wrong – el producer Oct 18 '14 at 13:39
  • About which buffer are you talking now? If your app has sent everything it can start reading. When the server has received everything it will respond. Your app can read the response as soon as it comes in. That's how it goes with http. – greenapps Oct 20 '14 at 11:00
  • @greenapps the buffer I keep referring to is the progressiveoutputstream that is written to in chunks – el producer Oct 21 '14 at 02:53

5 Answers5

1
class ProgressiveOutputStream extends ProxyOutputStream {
            long totalSent;
            public ProgressiveOutputStream(OutputStream proxy) {
                   super(proxy);
                   totalSent = 0;
            }

            public void write(byte[] bts, int st, int end) throws IOException {

            // FIXME  Put your progress bar stuff here!
            // end is the amount being sent this time
            // st is always zero and end=bts.length()

                 totalSent += end;
                 progress.publish((int) ((totalSent / (float) totalSize) * 100));
                 out.write(bts, st, end);
            }
Adnan Abdollah Zaki
  • 4,328
  • 6
  • 52
  • 58
0

Since I don't see any solution for this, I suppose the answer is to use a spinning animation without a progress percent. Since nothing can be done until the transmission is complete anyway. Oh well... atleast it solved my problem.

el producer
  • 321
  • 3
  • 9
  • I still think you were talking about a problem that does not exist. – greenapps Oct 21 '14 at 08:43
  • If I am imagining this, can you please explain why after my progress reaches 100%, the uploaded file has just begun to show up as only a fraction of the total file size on my remote server? And why the server response doesnt come back to android client until file on server has reached its total size? And why if I remove all server side file processing and return an immediate response, it still takes the same amount of time for the server response to reach the client. Maybe im wrong about it buffering but that is what appears to be the problem to me. – el producer Oct 22 '14 at 13:34
  • `why after my progress reaches 100%, the uploaded file has just begun to show up as only a fraction of the total file size on my remote server?` Those are caching and buffering issues. The one is ready filling buffers, the othere is busy emptying them. `why the server response doesnt come back to android client until file on server has reached its total size?` Because the server will only start responding when all is 'in'. The sever cannot even send something before it has read everything as that is forbidden by the http protocol I said that before. – greenapps Oct 22 '14 at 14:00
  • Yeah, exactly! Thanks for confirming – el producer Oct 24 '14 at 03:20
0

maybe you could flush the data every time you call write:

totalSent += end;
int progress = (int) ((totalSent / (float) totalSize) * 100);
out.write(bts, st, end);
out.flush(); //flush

Edit: you can also try this since i get the feeling end represents the end index in the outputstream named out:

totalSent += (end-st);
Maro
  • 253
  • 3
  • 8
  • thanks, i tried this but didn't notice any difference with a 92mb video. Actually i didn't notice any lag with or without the flush statement so can't say if this made any difference at all. – el producer Apr 12 '15 at 09:33
  • i get the feeling the value `end` isn't really the amount sent but the end index. I know it says it's the amout sent in the documentation but i think it's represented as the end index so that when you call `out.wrtite(bts, st, end);` it handles it correctly.I know it doesn't seem right but can you try this: `totalSent += (end - st);` instead – Maro Apr 25 '15 at 05:15
0

I know this is quite old, but I've just found the solution. If totalSize is your entity content length, then you can base your progress on that:

class ProgressiveOutputStream extends ProxyOutputStream {
                    long totalSent;
                    long totalSize;
                    public ProgressiveOutputStream(OutputStream proxy, long total) {
                        super(proxy);
                        totalSent = 0;
                        totalSize = total;
                    }
                    public void write(byte[] bts, int st, int end) throws IOException {
                        totalSent += end;
                        publishProgress((int) ((totalSent / (float) totalSize) * 100));
                        out.write(bts, st, end);
                    }
                }

                yourEntity.writeTo(new ProgressiveOutputStream(outstream, yourEntity.getContentLength()));

You update your progressbar in OnProgressUpdate of your asyncTask (pb is the progressBar):

@Override
    protected void onProgressUpdate(Integer... values) {
        super.onProgressUpdate(values);
        pb.setProgress(values[0]);
    }
Luchi Valles
  • 181
  • 4
  • thanks but unfortunately I used Thread instead of AsyncTask and I don't have time right now to try this out. But if i ever need to modify this piece I will give it a shot. – el producer Apr 12 '15 at 09:32
0

Please try next solution instead of using ProxyOutputStream in your writeTo method:

@Override 
public void writeTo(OutputStream outstream) throws IOException {
    ByteArrayInputStream reader = new ByteArrayInputStream(mImageData);
    byte[] fileBuffer = new byte[2048];
    int bytesRead;
    while ((bytesRead = reader.read(fileBuffer)) != -1) {
        outstream.write(fileBuffer, 0, bytesRead);
        int progress = bytesRead;
    }
    reader.close();
    yourEntity.writeTo(outstream);
}
novachevskyi
  • 173
  • 2
  • 8
  • Why try this solution instead of that one? – pete the pagan-gerbil Sep 10 '15 at 12:23
  • OP mentioned that his solution not works correctly. I've figured that his solution not counts sent bytes properly. If you will try you can notice that there will be only a half of total amount of bytes in final progress. My solution returns proper count of sent bytes which I'm getting from `reader.read(fileBuffer)` method. – novachevskyi Sep 10 '15 at 14:03