7

Im trying to publish upload progress for images, uploading them using okhttp as my client and mimecraft to package up the multipart file.

I have added logs to write the bytecount when data gets written to the socket (in chunks of 4kb if i can tell correctly) and then uploaded.

The problem is that, while i call flush on the outputstream every time a chunk of data is written, nothing seems to get uploaded until ~800kb has been written to the outputstream. Once it hits that point, it seems to upload ~100kb, and the app then writes another 100kb to the outputstream (like its freed up space and can write a bit more) and continues on.

The result of this is that (over a 1.2mb file) the first 800kb gets written/reported almost instantly, and then it starts uploading (tracked through charles network software) and will then start a read/write/upload of 100kb over the next few seconds, then once ive written the last bytes to the outputstream, the app reports 100% of the upload is written. This sint true though as the network client is still uploading the last 800kb that still is in the network buffer, and it sits there for another 5-10 seconds uploading that, then finishes the request.

Has anyone had this experience, or knows if this is a common problem in okhttp?

Cheers

EDIT: if i upload a file less than 800kb, ive tested some 250kb and 500kb, and they all get written 100% to the stream instantly before being uploaded, but a 3mb image will still upload 800kb, and then tick away in roughly 100kb chunks, and the writing loop sits there writing another 100kb each time some gets uploaded.

Jake Wharton
  • 75,598
  • 23
  • 223
  • 230
Glenn.nz
  • 2,261
  • 2
  • 17
  • 20
  • do you have any success. Please also share your uploading code. – AZ_ May 19 '14 at 07:51
  • Any news on this, I'm running into the same issue? – coalmee Nov 13 '15 at 22:56
  • @Glenn.nz I know its been a long time since you asked this question but did you find any good solution for this? It is still buffering the 200KB of data even though I have set the send buffer size to 16KB. – Vikalp Oct 07 '20 at 04:36

3 Answers3

3

As the answer from @nmr said its because the send buffer's are too high as explained by the bug ticket here: https://github.com/square/okhttp/issues/1078

So I've created a socket factory that lets you override the send buffer size.

https://gist.github.com/slightfoot/00a26683ea68856ceb50e26c7d8a47d0

Simon
  • 10,932
  • 50
  • 49
  • 1
    Hey @Simon, Override the send buffer size is not helping much. No matter what buffer size you set, it is still buffering 200KB of data upfront but it is better than the 800KB buffer I suppose :). – Vikalp Oct 07 '20 at 04:31
  • This is probably a change in the OS and its new built in integrations with OkHttp since I wrote this original code. – Simon Oct 08 '20 at 14:31
1

The socket send buffer is defaulting to a high value, and flush isn't hooked up. I have been unsuccessful trying to find a way to sync a Java socket, the default Socket OutputStream ignores flushes.

https://github.com/square/okhttp/issues/1078

Terrible hack fork setting the send buffer size here: https://github.com/ACMEAtronOmatic/okhttp/tree/topic/hack-to-fix-socket-sendbuffer-size

nmr
  • 16,625
  • 10
  • 53
  • 67
  • 1
    Here's a method that doesn't involve using a customized version of okHttp. https://gist.github.com/slightfoot/00a26683ea68856ceb50e26c7d8a47d0 – Simon Apr 04 '16 at 07:51
  • @simon This comment is pretty answer-y, maybe put it up as an answer? – nmr Apr 04 '16 at 20:04
-1

Try calling setFixedLengthStreamingMode(content length) or setChunkedStreamingMode(0) before you open the output stream.

Jesse Wilson
  • 39,078
  • 8
  • 121
  • 128
  • 1
    Ive called `connection.getOutputStream()` after calling `setFixedLengthStreamingMode()` with the correct content length. Its definitely using the content-length i set, as if i change it (add 2 bytes or whatever) the call fails, and it starts uploading before all of the bytes are written to the stream. – Glenn.nz May 12 '14 at 12:25
  • I still see the reported behavior after trying both of these – nmr Oct 10 '14 at 22:25