What is an Android Binder "Transaction?"

Question

I'm getting an TransactionTooLargeException when sending messages between two Android processes running from a single APK. Each message only contains small amounts of data, much smaller than the 1 mb total (as specified in the docs).

I created a test app (code below) to play around with this phenomenon, and noticed three things:

1. I got a android.os.TransactionTooLargeException if each message was over 200 kb.

2. I got a android.os.DeadObjectException if each message was under 200kb

3. Adding a Thread.sleep(1) seems to have solved the issue. I cannot get either exception with a Thread.sleep

Looking through the Android C++ code, it seems like the transaction fails for an unknown reason and interpreted as one of those exceptions

Questions

1. What is a "transaction"?
2. What defines what goes in a transaction? Is it a certain number of events in a given time? Or just a max number/size of events?
3. Is there a way to "Flush" a transaction or wait for a transaction to finish?
4. What's the proper way to avoid these errors? (Note: breaking it up into smaller pieces will simply throw a different exception)

---

---

Code

AndroidManifest.xml

<?xml version="1.0" encoding="utf-8"?>
<manifest package="com.example.boundservicestest"
          xmlns:android="http://schemas.android.com/apk/res/android">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN"/>

                <category android:name="android.intent.category.LAUNCHER"/>
            </intent-filter>
        </activity>
        <service android:name=".BoundService" android:process=":separate"/>
    </application>

</manifest>

MainActivity.kt

class MainActivity : AppCompatActivity() {

    private lateinit var sendDataButton: Button
    private val myServiceConnection: MyServiceConnection = MyServiceConnection(this)

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        myServiceConnection.bind()

        sendDataButton = findViewById(R.id.sendDataButton)

        val maxTransactionSize = 1_000_000 // i.e. 1 mb ish
        // Number of messages
        val n = 10
        // Size of each message
        val bundleSize = maxTransactionSize / n

        sendDataButton.setOnClickListener {
            (1..n).forEach { i ->
                val bundle = Bundle().apply {
                    putByteArray("array", ByteArray(bundleSize))
                }
                myServiceConnection.sendMessage(i, bundle)
                // uncommenting this line stops the exception from being thrown
//                Thread.sleep(1)
            }
        }
    }
}

MyServiceConnection.kt

class MyServiceConnection(private val context: Context) : ServiceConnection {
    private var service: Messenger? = null

    fun bind() {
        val intent = Intent(context, BoundService::class.java)
        context.bindService(intent, this, Context.BIND_AUTO_CREATE)
    }

    override fun onServiceConnected(name: ComponentName, service: IBinder) {
        val newService = Messenger(service)
        this.service = newService
    }

    override fun onServiceDisconnected(name: ComponentName?) {
        service = null
    }

    fun sendMessage(what: Int, extras: Bundle? = null) {
        val message = Message.obtain(null, what)
        message.data = extras
        service?.send(message)
    }
}

BoundService.kt

internal class BoundService : Service() {
    private val serviceMessenger = Messenger(object : Handler() {
        override fun handleMessage(message: Message) {
            Log.i("BoundService", "New Message: ${message.what}")
        }
    })

    override fun onBind(intent: Intent?): IBinder {
        Log.i("BoundService", "On Bind")
        return serviceMessenger.binder
    }
}

build.gradle*

apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'

android {
    compileSdkVersion 27
    defaultConfig {
        applicationId "com.example.boundservicestest"
        minSdkVersion 19
        targetSdkVersion 25
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
}

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
    implementation 'com.android.support:appcompat-v7:27.1.1'
}

Stacktrace

07-19 09:57:43.919 11492-11492/com.example.boundservicestest E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.example.boundservicestest, PID: 11492
    java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:448)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
     Caused by: java.lang.reflect.InvocationTargetException
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807) 
     Caused by: android.os.DeadObjectException: Transaction failed on small parcel; remote process probably died
        at android.os.BinderProxy.transactNative(Native Method)
        at android.os.BinderProxy.transact(Binder.java:764)
        at android.os.IMessenger$Stub$Proxy.send(IMessenger.java:89)
        at android.os.Messenger.send(Messenger.java:57)
        at com.example.boundservicestest.MyServiceConnection.sendMessage(MyServiceConnection.kt:32)
        at com.example.boundservicestest.MainActivity$onCreate$1.onClick(MainActivity.kt:30)
        at android.view.View.performClick(View.java:6294)
        at android.view.View$PerformClick.run(View.java:24770)
        at android.os.Handler.handleCallback(Handler.java:790)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:164)
        at android.app.ActivityThread.main(ActivityThread.java:6494)
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438) 
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807) 

Answer 1

1) What is a "transaction"?

When a client process makes a call to the server process (In our case service?.send(message)), it transfers a code representing the method to call along with marshalled data (Parcels). This call is called a transaction. The client Binder object calls transact() whereas the server Binder object receives this call in onTransact() method. Check This and This.

2) What defines what goes in a transaction? Is it a certain number of events in a given time? Or just a max number/size of events?

In General it is decided by Binder protocol.They make use of proxies (by client) and stubs (by service). Proxies take your high-level Java/C++ method calls (requests) and convert them to Parcels (Marshalling) and submit the transaction to the Binder Kernel Driver and block. Stubs on the other hand (in the Service process) listens to the Binder Kernel Driver and unmarshalls Parcels upon receiving a callback, into rich data types/objects that the Service can understand.

In case of Android Binder framwork send The data through transact() is a Parcel(It means that we can send all types of data supported by Parcel object.), stored in the Binder transaction buffer.The Binder transaction buffer has a limited fixed size, currently 1Mb, which is shared by all transactions in progress for the process. So if each message is over 200 kb, Then 5 or less running transactions will result in limit to exceed and throw TransactionTooLargeException. Consequently this exception can be thrown when there are many transactions in progress even when most of the individual transactions are of moderate size. An activity will see DeadObjectException exception if it makes use of a service running in another process that dies in the middle of performing a request. There are plenty of reasons for a process to kill in Android. Check this blog for more info.

3) Is there a way to "Flush" a transaction or wait for a transaction to finish?

A call to transact() blocks the client thread(Running in process1) by default until onTransact() is done with its execution in the remote thread(Running in process2).So the transaction API is synchronous in nature in Android. If you don’t want the transact() call to block then you can pass the IBinder.FLAG_ONEWAY flag(Flag to transact(int, Parcel, Parcel, int)) to return immediately without waiting for any return values.You have to implement your custom IBinder implementation for this.

4) What's the proper way to avoid these errors? (Note: breaking it up into smaller pieces will simply throw a different exception)

1. Limit no of transactions at a time. Do transactions which are really necessary(with message size of all ongoing transactions at a time must be less than 1MB).
2. Make sure process(other than app process) in which other Android component running must be running.

Note:- Android support Parcel to send data between different processes. A Parcel can contain both flattened data that will be unflattened on the other side of the IPC (using the various methods here for writing specific types, or the general Parcelable interface), and references to live IBinder objects that will result in the other side receiving a proxy IBinder connected with the original IBinder in the Parcel.

Proper way to bind a service with activity is bind service on Activity onStart() and unbind it in onStop(), which is visible life-cycle of an Activity.

In your case Add on method in MyServiceConnection class :-

fun unBind() { context.unbindService(this) }

And in your Activity class:-

override fun onStart() {
        super.onStart()
        myServiceConnection.bind()
    }

    override fun onStop() {
        super.onStop()
        myServiceConnection.unBind()
    }

Hope this will help you.

Answer 2

1. What is a "transaction"?

During a remote procedure call, the arguments and the return value of the call are transferred as Parcel objects stored in the Binder transaction buffer. If the arguments or the return value are too large to fit in the transaction buffer, then the call will fail and TransactionTooLargeException will be thrown.

2. What defines what goes in a transaction? Is it a certain number of events in a given time? Or just a max number/size of events? Only and only size The Binder transaction buffer has a limited fixed size, currently 1Mb, which is shared by all transactions in progress for the process.

3. Is there a way to "Flush" a transaction or wait for a transaction to finish?

No

4. What's the proper way to avoid these errors? (Note: breaking it up into smaller pieces will simply throw a different exception)

As per my understanding your message object might be having bytearray of image or something else which is having size more than 1mb. Don't send bytearray in Bundle.

Option 1: For image I think you should pass the URI through Bundle. Use Picasso as its uses caching wont download image multiple times.

Option 2 [not recommended] Compress byte array, because it might not compress upto required size

//Convert to byte array
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bmp.compress(Bitmap.CompressFormat.PNG, 100, stream);
byte[] byteArr = stream.toByteArray();

Intent in1 = new Intent(this, Activity2.class);
in1.putExtra("image",byteArr);

Then in Activity 2:

byte[] byteArr = getIntent().getByteArrayExtra("image");
Bitmap bmp = BitmapFactory.decodeByteArray(byteArr, 0, byteArr.length);

Option 3 [Recommended] use file read / write and pass the uri via bundle

Write File:

private void writeToFile(String data,Context context) {
    try {
        OutputStreamWriter outputStreamWriter = new OutputStreamWriter(context.openFileOutput("filename.txt", Context.MODE_PRIVATE));
        outputStreamWriter.write(data);
        outputStreamWriter.close();
    }
    catch (IOException e) {
        Log.e("Exception", "File write failed: " + e.toString());
    } 
}

Read File:

private String readFromFile(Context context) {

    String ret = "";

    try {
        InputStream inputStream = context.openFileInput("filename.txt");

        if ( inputStream != null ) {
            InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
            BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
            String receiveString = "";
            StringBuilder stringBuilder = new StringBuilder();

            while ( (receiveString = bufferedReader.readLine()) != null ) {
                stringBuilder.append(receiveString);
            }

            inputStream.close();
            ret = stringBuilder.toString();
        }
    }
    catch (FileNotFoundException e) {
        Log.e("login activity", "File not found: " + e.toString());
    } catch (IOException e) {
        Log.e("login activity", "Can not read file: " + e.toString());
    }

    return ret;
}

Option 4 (Using gson) Write object

[YourObject] v = new [YourObject]();
Gson gson = new Gson();
String s = gson.toJson(v);

FileOutputStream outputStream;

try {
  outputStream = openFileOutput(filename, Context.MODE_PRIVATE);
  outputStream.write(s.getBytes());
  outputStream.close();
} catch (Exception e) {
  e.printStackTrace();
}

How to read it back:

 FileInputStream fis = context.openFileInput("myfile.txt", Context.MODE_PRIVATE);
 InputStreamReader isr = new InputStreamReader(fis);
 BufferedReader bufferedReader = new BufferedReader(isr);
 StringBuilder sb = new StringBuilder();
 String line;
 while ((line = bufferedReader.readLine()) != null) {
     sb.append(line);
 }

 String json = sb.toString();
 Gson gson = new Gson();
 [YourObject] v = gson.fromJson(json, [YourObject].class);

Answer 3

Others are much explained the situation, but there is some important things missing.

1) What is a "transaction"? Transactions are predefined messages, that passes buffer through interprocess barrier (with help of kernel - implementation is inside kernel - https://elixir.bootlin.com/linux/latest/source/drivers/android/binder.c - Not the C++/Java user code.). It uses serialized buffers named parcels, which gets passed through kernel for the consumers. Transactions are meant for small self contained data, not huge memory buffers.

2) What defines what goes in a transaction? Is it a certain number of events in a given time? Or just a max number/size of events? Transaction are more or less writes to kernel to "transmit" via bus to connected clients. It seems that you have limit.

3) Is there a way to "Flush" a transaction or wait for a transaction to finish? Depends, you shouldn't need this.

4) What's the proper way to avoid these errors? (Note: breaking it up into smaller pieces will simply throw a different exception)

Design your application slightly more reasonably: If you are in the same memory you can share object.

However there is one overlooked feature that allows to share large amount of data using binder: you can map entire file descriptors. There is a class: https://developer.android.com/reference/android/os/ParcelFileDescriptor

You can use to wrap memory files: https://developer.android.com/reference/android/os/MemoryFile

In general file descriptor when passed can be loaded by retrieving process as android behind the scenes maps those descriptors as accessible to the passed process. This is the actual "fix". File descriptor itself can have huge buffer attached.