Android - how to upload video in chunks using OkHTTP?

Question

please see my below code which I use to upload a video to a server. However, for large enough videos, I'm getting an OutOfMemory exception.

        InputStream stream = getContentResolver().openInputStream(videoUri);
        byte[] byteArray = IOUtils.toByteArray(stream);
        RequestBody requestBody = new MultipartBody.Builder()
                .setType(MultipartBody.FORM)
                .addFormDataPart("file", "fname",
                        RequestBody.create(MediaType.parse("video/mp4"), byteArray))
                .build();
        Request request = new Request.Builder()
                .url(uploadURL)
                .post(requestBody)
                .build();

        OkHttpClient client = new OkHttpClient.Builder().build();
        client.newCall(request).enqueue(new Callback() {
            @Override
            public void onFailure(Call call, IOException e) {
            }
            @Override
            public void onResponse(Call call, Response response) throws IOException {
            }
        });

Can someone point me in the right direction on how to avoid OutOfMemory exception? Is there a way I can go from the InputStream to the requestBody?

score 8 · Accepted Answer · answered Jul 16 '17 at 00:39

You can create a custom RequestBody which streams the data. You will have to take some care: it may be reused multiple times because OkHttp may decide to retry the request. Make sure you can re-open the InputStream from the start each time.

ContentResolver contentResolver = context.getContentResolver();
final String contentType = contentResolver.getType(videoUri);
final AssetFileDescriptor fd = contentResolver.openAssetFileDescriptor(videoUri, "r");
if (fd == null) {
    throw new FileNotFoundException("could not open file descriptor");
}
RequestBody videoFile = new RequestBody() {
    @Override public long contentLength() { return fd.getDeclaredLength(); }
    @Override public MediaType contentType() { return MediaType.parse(contentType); }
    @Override public void writeTo(BufferedSink sink) throws IOException {
        try (InputStream is = fd.createInputStream()) {
            sink.writeAll(Okio.buffer(Okio.source(is)));
        }
    }
};
RequestBody requestBody = new MultipartBody.Builder()
        .setType(MultipartBody.FORM)
        .addFormDataPart("file", "fname", videoFile)
        .build();
Request request = new Request.Builder()
        .url(uploadURL)
        .post(requestBody)
        .build();
client.newCall(request).enqueue(new Callback() {
    @Override public void onFailure(Call call, IOException e) {
        try {
            fd.close();
        } catch (IOException ex) {
            e.addSuppressed(ex);
        }
        Log.e(TAG, "failed", e);
    }
    @Override public void onResponse(Call call, Response response) throws IOException {
        fd.close();
    }
});

Thank you so much! Do you know how I can re-open the stream in case the request is retried? — JK140, Jul 16 '17 at 00:56
Also, what is the purpose of calling fd.close() only after a response is received? — JK140, Jul 16 '17 at 00:57
@JK140 1. Open an `InputStream` each time `RequestBody.writeTo()` is invoked, like this example does. If you open the `InputStream` outside of `writeTo()`, it will read the whole thing the first time and subsequent retries will have no data. 2. The `AssetFileDescriptor` continues to be used until the response is complete (and it must be closed, otherwise it's a resource leak). You could also *not* use `AssetFileDescriptor`, and instead fetch the length once (perhaps using `ContentResolver.query()`), and use and close `ContentResolver.openInputStream()` inside `writeTo`. — ephemient, Jul 16 '17 at 01:03
Really a great answer. Here is a footnote for those who doesn't use transfer-encoding:chunked to transmit data: you just need to override the contentLength() method and OKHttp will automatically add Content-Length header to the request. fd.getDeclaredLength() will return -1 so the transfer-encoding is still "chuked"; While fd.getLength() will return the file's actual size so chunked is disabled. — vainquit, Feb 01 '21 at 01:13

Jemshit · Answer 2 · 2021-04-26T09:14:34.617

Tries to convert Uri to InputStream using one of AssetFileDescriptor/ ContentResolver/ ParcelFileDescriptor
Creates okio.Source from InputStream (about source/sink)
Write Source into Sink, then close InputStream

If you know what server expects as contentType, hardcode it. Because server might only accept application/octet-stream even if file is video/mp4. Uri to contentType

Check for Uri to contentLength, if contentLength is not found, Content-Length: X header will not be appended when uploading.

/** It supports file/content/mediaStore/asset URIs. asset not tested */
fun createAssetFileDescriptor() = try {
    contentResolver.openAssetFileDescriptor(this, "r")
} catch (e: FileNotFoundException) {
    null
}

/** It supports file/content/mediaStore URIs. Will not work with providers that return sub-sections of files */
fun createParcelFileDescriptor() = try {
    contentResolver.openFileDescriptor(this, "r")
} catch (e: FileNotFoundException) {
    null
}

/** - It supports file/content/mediaStore/asset URIs. asset not tested
 * - When file URI is used, may get contentLength error (expected x but got y) error when uploading if contentLength header is filled from assetFileDescriptor.length */
fun createInputStreamFromContentResolver() = try {
    contentResolver.openInputStream(this)
} catch (e: FileNotFoundException) {
    null
}

fun Uri.asRequestBody(contentResolver: ContentResolver,
                      contentType: MediaType? = null,
                      contentLength: Long = -1L)
        : RequestBody {

    return object : RequestBody() {
        /** If null is given, it is binary for Streams */
        override fun contentType() = contentType

        /** 'chunked' transfer encoding will be used for big files when length not specified */
        override fun contentLength() = contentLength

        /** This may get called twice if HttpLoggingInterceptor is used */
        override fun writeTo(sink: BufferedSink) {
            val assetFileDescriptor = createAssetFileDescriptor()
            if (assetFileDescriptor != null) {
                // when InputStream is closed, it auto closes AssetFileDescriptor
                AssetFileDescriptor.AutoCloseInputStream(assetFileDescriptor)
                        .source()
                        .use { source -> sink.writeAll(source) }
            } else {
                val inputStream = createInputStreamFromContentResolver()
                if (inputStream != null) {
                    inputStream
                            .source()
                            .use { source -> sink.writeAll(source) }
                } else {
                    val parcelFileDescriptor = createParcelFileDescriptor()
                    if (parcelFileDescriptor != null) {
                        // when InputStream is closed, it auto closes ParcelFileDescriptor
                        ParcelFileDescriptor.AutoCloseInputStream(parcelFileDescriptor)
                                .source()
                                .use { source -> sink.writeAll(source) }
                    } else {
                        throw IOException()
                    }
                }
            }
        }
    }
}

Usage:

val request = uri.asRequestBody(
       contentResolver = context.contentResolver,
       contentType = "application/octet-stream".toMediaTypeOrNull(),
       contentLength = uri.length(context.contentResolver)
)

If you move local functions from the `fun` body to anonymous class, you will save up some bytecode. — Miha_x64, Apr 25 '21 at 17:48

Android - how to upload video in chunks using OkHTTP?

2 Answers2

Linked