0

I have a library written in C++ that has APIs to transfer bulk data, both via copy and move semantics. For ex. one of the function signature looks like this:

template <
    typename ITERATOR,
    typename TYPE = typename std::iterator_traits<ITERATOR>::value_type>
    void appendData(std::size_t  columnIndex,
                    ITERATOR      begin,
                    ITERATOR      end);

A caller can pass in an iterator to a data structure like a std::vector and this method will append the data starting from 'begin' up to but not including the 'end' iterator. The caller can also pass in a std::move_iterator instead of a regular iterator, and the data would be moved into the internal structures instead of being copied. The recommendation to the users of this library is to pass in a std::move_iterator for efficiency reasons.

I want to access this library from Java code (possibly via JNI) and use this interface to transfer data. The question is how do I transfer ownership of a buffer of data that was allocated inside Java to the C++ side so that the Java allocated buffer of data is later freed in C++?

I thought about using apache arrow as the data transfer mechanism, but again the ownership of the arrow structures still rests with the (Java) code that allocated these structures if I am not wrong.

Ashvakan
  • 1
  • 1
  • Typically the incoming data is appended to an internal structure and it is sorted based on some criteria, before being sent upstream on the wire through a data transfer mechanism. Now that I think of it, would it make sense to keep the ownership in Java and just keep the data buffer from being garbage collected until the C++ layer is done using it. – Ashvakan Mar 17 '22 at 15:55
  • Typical zero-magic approach when Language B needs to release memory allocated by Language A is to provide a function that Language B calls when it's done using the storage. Inside that function you can do all sorts of magic, like find the allocation in and remove the allocation from a container class that keeps the allocation alive. – user4581301 Mar 17 '22 at 15:58
  • Use `NewDirectByteBuffer` in JNI. Now the memory is owned by C++ and can be used in Java and C++. – Brandon Mar 17 '22 at 16:59
  • @Brandon [Direct byte buffers are not that easy to use properly](https://stackoverflow.com/questions/28791827/does-newdirectbytebuffer-create-a-copy-in-native-code). Until actual performance profiling indicates that copying data between native memory and Java `byte[]` (or other type) arrays is a demonstrated performance bottleneck, just copy the data when needed. The code is simpler, which is a ***huge*** advantage when writing JNI code given how fragile JNI can be. The data is "owned" by only one object, be it C++ or Java, and copied when needed. – Andrew Henle Mar 19 '22 at 13:20

0 Answers0