rxandroidble write only sends the first 20B

Question

I try to write >20 bytes data on a given (custom) characteristic. In the following log, I tried to write 85 bytes:

code:

connectionObservable
                .flatMap(rxBleConnection -> rxBleConnection.writeCharacteristic(
                        wChar.uuid(),
                        wChar.bytes()))
                .observeOn(mainThread())
                .subscribe(
                        bytes -> wChar.success(),
                        this::onWriteFailure
                );

result: On the server side (nrf52) I can see the EXEC_WRITE but only the first 20B are sent.

this is the logcat:

D/RxBle#ClientOperationQueue: QUEUED ConnectOperation(17461182) D/RxBle#ClientOperationQueue: STARTED ConnectOperation(17461182) D/RxBle#ClientOperationQueue: QUEUED ConnectOperation(218660306) D/RxBle#ClientOperationQueue: STARTED ConnectOperation(218660306) D/RxBle#BluetoothGatt: onConnectionStateChange newState=2 status=0 D/RxBle#BluetoothGatt: onConnectionStateChange newState=2 status=0 D/RxBle#ClientOperationQueue: FINISHED ConnectOperation(218660306) D/RxBle#ClientOperationQueue: FINISHED ConnectOperation(17461182) D/RxBle#ConnectionOperationQueue: QUEUED ServiceDiscoveryOperation(125599796) D/RxBle#ConnectionOperationQueue: STARTED ServiceDiscoveryOperation(125599796) D/RxBle#BluetoothGatt: onServicesDiscovered status=0 D/RxBle#ConnectionOperationQueue: QUEUED CharacteristicReadOperation(2626026) D/RxBle#ConnectionOperationQueue: FINISHED ServiceDiscoveryOperation(125599796) D/RxBle#ConnectionOperationQueue: STARTED CharacteristicReadOperation(2626026) D/RxBle#BluetoothGatt: onCharacteristicRead characteristic=0000fa03-0278-03be-4447-091eba91df8e status=0 D/RxBle#ConnectionOperationQueue: FINISHED CharacteristicReadOperation(2626026) D/RxBle#ClientOperationQueue: QUEUED ConnectOperation(158692575) D/RxBle#ClientOperationQueue: STARTED ConnectOperation(158692575) D/RxBle#BluetoothGatt: onConnectionStateChange newState=2 status=0 D/RxBle#ClientOperationQueue: FINISHED ConnectOperation(158692575) D/RxBle#ConnectionOperationQueue: QUEUED ServiceDiscoveryOperation(20778996) D/RxBle#ConnectionOperationQueue: STARTED ServiceDiscoveryOperation(20778996) > D/RxBle#BluetoothGatt:onServicesDiscovered status=0
D/RxBle#ConnectionOperationQueue: QUEUED CharacteristicWriteOperation(51009974) D/RxBle#ConnectionOperationQueue: FINISHED ServiceDiscoveryOperation(20778996) D/RxBle#ConnectionOperationQueue: STARTED CharacteristicWriteOperation(51009974)
> D/RxBle#BluetoothGatt: onCharacteristicWrite characteristic=0000fa04-0278-03be-4447-091eba91df8e status=0

D/RxBle#ConnectionOperationQueue: FINISHED CharacteristicWriteOperation(51009974)

I also tried to use the long rxAndroidBlewrite procedure:

connectionObservable
                .flatMap(rxBleConnection -> {
                            rxBleConnection.setupNotification(wChar.uuid()); 
                            return rxBleConnection.createNewLongWriteBuilder()
                                    .setCharacteristicUuid(wChar.uuid()) 
                                    .setBytes(array)
                                    .build();
                        }
                )
                .subscribe(
                        bytes -> wChar.success(),
                        this::onWriteFailure
                );

and it sends several successive write commands but it is not the long write procedure (with n ATT_prepare and 1 ATT_exec), it's independant writes:

D/RxBle#ConnectionOperationQueue: QUEUED CharacteristicLongWriteOperation(74131396) D/RxBle#ConnectionOperationQueue: FINISHED ServiceDiscoveryOperation(250008320) D/RxBle#ConnectionOperationQueue: STARTED CharacteristicLongWriteOperation(74131396)

D/RxBle#BluetoothGatt: onCharacteristicWrite characteristic=0000fa04-0278-03be-4447-091eba91df8e status=0

D/RxBle#BluetoothGatt: onCharacteristicWrite characteristic=0000fa04-0278-03be-4447-091eba91df8e status=0

D/RxBle#BluetoothGatt: onCharacteristicWrite characteristic=0000fa04-0278-03be-4447-091eba91df8e status=0

D/RxBle#BluetoothGatt: onCharacteristicWrite characteristic=0000fa04-0278-03be-4447-091eba91df8e status=0

D/RxBle#BluetoothGatt: onCharacteristicWrite characteristic=0000fa04-0278-03be-4447-091eba91df8e status=0

D/RxBle#ConnectionOperationQueue: FINISHED CharacteristicLongWriteOperation(74131396)

of course I could manage to rebuild at the server or to modify the MTU, but I want to use the BLE queued writes, which is normally supported by my central (rxandroidble) and my peripheral (nrf52)

[related post](https://devzone.nordicsemi.com/question/186420/long-write-receiving-partial-data-only/) on nordic devzone — yacine, Jan 11 '18 at 22:06
thanks I'm currently finishing the migration towards Nordic SDK14 and will test again. I'll post the results — yacine, Jan 31 '18 at 15:11

score 1 · Answer 1 · answered Jan 11 '18 at 22:48

1

The Bluetooth 4.0 spec, which included the introduction of BLE, states that a maximum of 20 bytes can be transferred on a given characteristic at a time. If you need to send more data, you will have to send 20 bytes at a time in some type of loop.

So in fact, this isn't an issue with RxAndroidBle, just a limitation of the technology.

See here: https://stackoverflow.com/a/38914831/4321774

answered Jan 11 '18 at 22:48

vdelricco

749
4
15

thanks but i think this is not true: there is a way in BLE to manage long writes:https://community.nxp.com/thread/332032#441552 – yacine Jan 15 '18 at 15:32
Partly true. On the ATT layer you can only send 20 bytes (or what the MTU allows for a particular operation), but the GATT layer (which Android implements) defines procedures to atomically perform longer writes using several ATT packets. So when you do a normal Write characteristic in Android it will use this method automatically when the value is too long for a single packet. – Emil Jan 24 '18 at 22:53
Fascinating, I did not know that. Thank you for responding @Emil – vdelricco Jan 25 '18 at 00:02

Dariusz Seweryn · Answer 2 · 2018-01-25T13:29:30.380

1

If you refer to the queued writes then on the Android API it seems to be referenced as reliable write. This API is not currently implemented in the RxAndroidBle and you would need to do that by implementing RxBleCustomOperation API using a shortcut to native BluetoothGattCallback. Even then it appears that the native Android API is not fully functional in this matter.

The RxAndroidBle long write is not using the prepared writes but multiple standard writes. This actually could be better described in the Javadoc...

There are mixed opinions on what a Long Write really is. @Emil's excellent answer in this question clarifies it very well.

I have performed some tests using an nRF51822 with Softdevice S110 from SDK 8.1.0.

It seems that under the hood a Long Write is just a Prepared Write- Android manages it for the user.

On the peripheral side it seems to be trickier to implement as the Softdevice informs the app that the Prepared Write has finished and that data is ready to be parsed (it is not attached to the write BLE event itself). Parsing of he data belongs to the app logic as there seems to be no distinction between a Long Write and a Prepared/Reliable Write which may take into consideration writing to more than one characteristic at a time and that there may be some business logic related consistency issues (whether a specific set of writes should be accepted or not).

Conclusion: Android vanilla API (and RxAndroidBle) does support so called Long Write out of the box by making multiple Prepared/Queued writes under the hood. It is up to the peripheral's firmware to handle it properly

edited Jan 25 '18 at 13:29

answered Jan 11 '18 at 23:15

Dariusz Seweryn

3,212
2
14
21

hello Dariusz (I'm a fan :-)), I have read this nordic post already several times. it says amoogn other things that **On Android it is enough to call writeCharacteristic(...) with data longer than 20 bytes to make it work. It should work on nRF Master Control Panel. Android will take care of it automatically.** and this is precisely why I dont understand why only 20B are sent when I ask android (via rxandroidble) to write 85B. – yacine Jan 12 '18 at 10:13
it says also that **The firmware must be prepared to received long write**, which I did and I think it works well because I succeed in retrieving the first 20B through the correct procedure. – yacine Jan 12 '18 at 10:16
concerning the **queued vs. prepared vs. long** terminology, I m completely lost after several weeks reading the forums. all I need to do is to send e.g. 100B long data on one given characteristic. – yacine Jan 12 '18 at 10:22
**The RxAndroidBle long write is not using the prepared writes but multiple standard writes. This actually could be better described in the Javadoc...** this is what I observe. see my second log extract with the multiple consecutive writes. but if i do this i cannot leverage on the nordic stack to manager the reassembly. – yacine Jan 12 '18 at 10:25
There seems to be an issue either on the firmware side or on the Android BLE stack side. The `RxAndroidBle` when using `RxBleConnection.writeCharacteristic()` does not alter the passed `byte[]` and it pass it (at some point) to the native `BluetoothGattCharacteristic`. If both Android and firmware side work correctly then everything should work. My virtual machine for nRF development just died but I will try to implement the long write on nRF51 and check it out. – Dariusz Seweryn Jan 12 '18 at 15:11
hello dariusw, sorry to ask. but I'm still stucked with this problem. – yacine Jan 19 '18 at 11:22
I didn't managed to test it this week although I've fixed an OSX toolchain for test so I should be able to do it next week – Dariusz Seweryn Jan 19 '18 at 16:35
@DariuszSeweryn see my answer https://stackoverflow.com/a/48432975/556495 which should clarify your confusion about the terminology. – Emil Jan 24 '18 at 22:49
@Emil thank you very much for confirming my suspicion and for an excellent answer. @yacine I have made a test with `RxAndroidBle` and an `nRF51` with `S110` and a Long Write worked out of the box on Android side. Handling it on the peripheral's side is a bit more cumbersome (the need of providing a bigger buffer and parsing the received data) but it is a different topic. I have edited my answer. – Dariusz Seweryn Jan 25 '18 at 13:34
Hello @DariuszSewery it works ! It means following Emil s' advices. I think it means that Android (and then RxAndroidBle) supports the long write silently. – yacine Feb 07 '18 at 09:56
Good to hear that! Yes, Android API (and RxAndroidBle) use this queued writes functionality in a transparent way (as I referenced it `out of the box`). Have a good day! :) – Dariusz Seweryn Feb 07 '18 at 10:03

score 1 · Accepted Answer · answered Jan 24 '18 at 22:47

For the client side (Android), just use the standard Write procedure and you will be fine there. Internally, it will split it up into multiple Prepare Write Requests followed by an Execute Write Request. That is, use your first approach.

To help with the confusion, there are two layers: GATT and ATT. The ATT layer defines a concept called "Queued Writes" which consists of Prepare Write Request/Response and Execute Write Request/Response. The idea is that all Prepared Writes (each of these are limited to MTU-5 in size, have an offset parameter and an ATT handle) are put in a queue at the peripheral side. The writes are not committed at the peripheral until the Execute Write occurs. (The Execute Write has a flag which can also be used to cancel the whole queue.)

On the GATT layer, we have something called "Write Long Characteristic Values". This is a procedure which will be used when a Characteristic value should be written that exceeds what can be put in a single Write Request. This procedure is defined to use the "Queued Writes" features in ATT, so it will split the value into multiple segments, send them all in Prepared Write packets and finally the Execute Write packet will be sent. "Write Long Characteristic Values" should be avoided at all costs due to its large overhead (one roundtrip per packet, the response packets are long because they contain a copy of the value, and one final Execute Write Request is needed). Instead increasing the MTU to the max is much better since that can send everything in a single connection event if you are lucky.

Reliable Writes is also a GATT layer feature that uses the "Queued Writes" feature in ATT. The idea is that the user should be able to perform multiple atomic writes to potentially more than one characteristic in one operation. The word reliable comes from that the client is supposed to be verifying that the values were sent correctly. Each Prepared Write Response include the received value from the Prepared Write Request and the client should compare these and see that they are equal, and if not, abort. Now in Android, it's impossible with the current API+implementation to perform this check, so while the operation still works, it is really not as "reliable" as it is supposed to be (but you have CRC on all BLE packets anyway so I don't think there would be an issue). Note that if you follow the GATT rules you may only perform Reliable Writes to Characteristics (not Descriptors) and only the Characteristics declaring this property.

On the peripheral side, it's impossible to really know if the incoming Prepared Write Requests are part of a "Write Long Characteristic Values" operation or a "Reliable Write" operation. But anyway, most BLE stacks don't combine the retrieved portions and then delivers a single Write when the Execute Write is received. They instead expose a quite low level API in my opinion; often just more or less forward the ATT packets.

For Nordic Semiconductor's softdevice, the easiest method they have is to use "GATTS Queued Writes: Stack handled, no attributes require authorization" http://infocenter.nordicsemi.com/topic/com.nordic.infocenter.s132.api.v5.0.0/group___b_l_e___g_a_t_t_s___q_u_e_u_e_d___w_r_i_t_e___b_u_f___n_o_a_u_t_h___m_s_c.html. This way it queues up all Prepared Writes in your application-provided buffer and notifies the app when the Execute Write arrives. The app should then parse all (more or less) raw ATT Prepared Write Request packets the stack has put in the buffer. This structure is defined here http://infocenter.nordicsemi.com/index.jsp?topic=%2Fcom.nordic.infocenter.s132.api.v5.0.0%2Fgroup___b_l_e___g_a_t_t_s___q_u_e_u_e_d___w_r_i_t_e_s___u_s_e_r___m_e_m.html&cp=2_3_1_1_0_2_4_5. NOTE that the buffer is a list (an array with these structures concatenated) and not a single value. The list is terminated with an item containing BLE_GATT_HANDLE_INVALID as handle. I think your mistake is that you only parse the first item in this list.

Thank you Emil, this is indeed crucial and did not found the info elsewhere. thanks ! the wiorking code is here: https://devzone.nordicsemi.com/f/nordic-q-a/29231/long-write-receiving-partial-data-only — yacine, Feb 07 '18 at 09:58

rxandroidble write only sends the first 20B

3 Answers3