How do I do S3 multipart upload from Android

Question

Overview: I have an app that uploads images and small audio files to S3 using the following approach:

1. Get a presigned URL from the backend using a secure API.
2. Upload to the presigned URL using Retrofit.

Problem: Now I need to support larger files like videos, so I am looking into implementing multi-part uploads.

Previous Solution: In a web app I did a while back, I had used the Uppy S3 Multipart plugin to handle multi-part uploads. The plugin had callbacks through which I could provide it with signed URLs for the individual parts. Everything else (splitting into parts, uploading) was handled by the plugin.

Current Situation: I have been searching for days for a viable alternative to Uppy S3 Multipart in Android but have not found one. Here are my findings:

1. There are no similar libraries for Android.
2. The official AWS docs suggest using AWS Amplify and make use of Cognito in all sample codes, which I don’t have in my project.
3. Some examples show using the AWS-Android-SDK to create an S3Client using credentials stored in the app, which is also not an option for me.

The farthest I've gotten: The option that looked the most promising is the TransferUtility class in the AWS Mobile SDK:

val transferUtility =
   TransferUtility.builder()
      .context(c)
      .awsConfiguration(awsConfiguration)
      .s3Client(s3Client)
      .build()

But where do I get the awsConfiguration and s3Client from? Also is BasicSessionCredentials class somehow related to my problem?

Questions: Is there an actual way to get this started? Is the only way to do this to manually handle the splitting and uploading? Why isn’t there an Uppy alternative yet in Android?

Answer 1

Regarding your current scenario, the TransferUtility class from the AWS Mobile SDK can indeed be used to handle the migration of multiple components to S3. To use this class, you will need to have the awsConfiguration and s3Client objects.

The AWSConfiguration class can be used to create the awsConfiguration object, which is usually located in your app's res/raw directory. This file contains your AWS configuration settings, such as S3 bucket names and fields. You can create an AWSConfiguration example like this:

val awsConfiguration = AWSConfiguration(applicationContext)

Next, you need to create an instance of s3Client using the AWS Mobile SDK. s3Client is an instance of the AmazonS3Client class, which you can create as follows.

val s3Client = AmazonS3Client(AWSMobileClient.getInstance())

Please note that you will need to add AWS Mobile SDK requirements and dependencies to your project.

Now, if you want to initiate a multipart upload using TransferUtility, you can use the upload() method. Here is an example of how you can use it.

val transferUtility = TransferUtility.builder()
    .context(applicationContext)
    .awsConfiguration(awsConfiguration)
    .s3Client(s3Client)
    .build()

val uploadObserver = transferUtility.upload(bucketName, objectKey, file)

In summary, you can use the TransferUtility class from the AWS Mobile SDK to handle multipart uploads in your Android app. Obtain the awsConfiguration and s3Client objects as described above, then use the upload() method of TransferUtility to start the upload process.

Answer 2

I ended up solving this using AWS Security Token Service. I fetch temporary credentials from our Oauth secured api, avoiding the need to store credentials in the app itself. Then I use the TransferUtility like this:

val secrets = //fetch from API

val awsCredentials: AWSCredentials = BasicSessionCredentials(secrets.accessKey, secrets.secretKey, secrets.sessionToken)
val s3Client = AmazonS3Client(awsCredentials, Region.getRegion(Regions.AP_SOUTHEAST_1))
val awsConfiguration = AWSConfiguration(context)
val transferUtility = TransferUtility.builder().context(context).awsConfiguration(awsConfiguration).s3Client(s3Client).build()
val uploadObserver: TransferObserver = transferUtility.upload(media.key, media.file)

Hope this helps anyone coming here in the future.