Flutter C++ Memory allocation causes jank on raster thread - Android NDK Dart FFI

Question

I have a flutter app which uses Dart ffi to connect to my custom C++ audio backend. There I allocate around 10MB of total memory for my audio buffers. Each buffer has 10MB / 84 of memory. I use 84 audio players. Here is the ffi flow:

C++ bridge:

extern "C" __attribute__((visibility("default"))) __attribute__((used))
void *
loadMedia(char *filePath, int8_t *mediaLoadPointer, int64_t *currentPositionPtr, int8_t *mediaID) {
    LOGD("loadMedia %s", filePath);

    if (soundEngine == nullptr) {
        soundEngine = new SoundEngine();
    }

    return soundEngine->loadMedia(filePath, mediaLoadPointer, currentPositionPtr, mediaID);
}

In my sound engine I launch a C++ thread:

void loadMedia(){

    std::thread{startDecoderWorker,
                    buffer,
    }.detach();
 }

void startDecoderWorker(float*buffer){
     buffer = new float[30000]; // 30000 might be wrong here, I entered a huge value to just showcase the problem, the calculation of 10MB / 84 code is redundant to the code
}

So here is the problem, I dont know why but when I allocate memory with new keyword even inside a C++ thread, flutters raster thread janks and I can see that my flutter UI janks lots of frames. This is also present in performance overlay as it goes all red for 3 to 5 frames with each of it taking around 30 40ms. Tested on profile mode.

Here is how I came to this conclusion: If I instantly return from my startDecoderWorker without running new memory allocation code, when I do this there is 0 jank. Everything is smooth 60fps, performance overlay doesnt show me red bars.

Here are some screenshots from Profile mode:

Have you tried just doing your memory allocation at program startup instead of thread startup? If the flutter thread is doing any sort of memory allocation, I could see there being some sort of lock contention in malloc: https://stackoverflow.com/questions/4600208/is-memory-allocation-in-linux-non-blocking Based on what I'm reading there, your threads could block for a long time while the allocator finds enough contiguous blocks to meet your request, which could block new/malloc in your flutter thread. — George, Mar 09 '21 at 19:01
I could do the thing you said but it would cause lots of logic since I keep on creating deleting audio players. This would require a lot of work for me and it seems like an anti pattern — cs guy, Mar 09 '21 at 19:16
For real-time software, it absolutely is NOT an anti-pattern. — George, Mar 09 '21 at 19:20
What if I suddenly need more memory? It would cause another jank this is why I said it's an anti pattern, I might be wrong. So it appears that my the best possibility for me to do this is to allocate memory at the very beginning of my app — cs guy, Mar 09 '21 at 19:36
Can you give a minimal reproducible sample (e.g. on GitHub)? Maybe it is *not* a problem for your `new float[30000]`, but caused by something else in your code! — ch271828n, Mar 27 '21 at 10:06
In addition, 10MB/84 sounds like a *tiny* thing. I myself deal with images in my app, and it is often a few megabytes (you know, uncompressed bitmaps). I do not see any junk. — ch271828n, Mar 27 '21 at 10:07
Actually your "If I instantly return from my startDecoderWorker without running new memory allocation..." logic may be wrong. If you directly return, that is an empty function, and C++ can optimize heavily on it. There are thousands of possible reasons without seeing your reproducible minimal sample. — ch271828n, Mar 27 '21 at 10:09
I tried to set the buffer size as 1 to test what you said. No jank. I think this is also another proof — cs guy, Mar 27 '21 at 12:33
i will provide an example soon `I myself deal with images in my app, and it is often a few megabytes (you know, uncompressed bitmaps).` did u use flutter for this? — cs guy, Mar 27 '21 at 16:17
@ch271828n added profile mode debug screenshots, I tried to create a simple project but I could not reproduce the jank. In the demo project In debug mode I was able to see performance hits but in profile mode i saw 0 hits. Bear in mind that my app is a complicated app, this bug is maybe due to several other things adding up BUT HOWEVER if I allocate only 2 floats instead of 30000 there is 0 jank. This should be enough to justify that there is a problem with mem alloc — cs guy, Mar 28 '21 at 01:52
@csguy that may not be a problem caused by 30000. for example, if you do something like `for(int i=0;i — ch271828n, Mar 28 '21 at 02:09
`did u use flutter for this?` -> c++ code (opencv), and use ffi to communicate w/ flutter — ch271828n, Mar 28 '21 at 02:10
@ch271828n you are a genius! I solved the issue, the problem was that its not because im allocating memory, its because as soon as I allocate memory my audio decoder starts decoding and fills the buffer. So apparently, its the decoder thats making everything lag. Even tho its in a seperate thread... its all clear now, thank you for chatting and helping me, make a comment and i will accept ur answer! — cs guy, Mar 28 '21 at 02:32

score 3 · Accepted Answer · answered Mar 28 '21 at 03:15

The actual cause, after discussions (in the comments of the question), is not because the memory allocation is too slow, but lie somewhere else - the calculations which will be heavy if the allocation is big.

For details, please refer to the comments and discussions of the question ;)

Flutter C++ Memory allocation causes jank on raster thread - Android NDK Dart FFI

1 Answers1