Using Android Nougat's shared memory for media decoding

Question

I am porting my Android AOSP-based distribution from Android K to Android N. It includes a modified version of the Media Player that decodes DVD subtitles. The architecture of the Media Player evolved a lot between those 2 versions. In particular, it is now split into 3 processes (see https://source.android.com/devices/media/framework-hardening).

I am thus trying to use Shared Memory to make the MediaCodecService send decoded bitmap subtitles to the MediaServer. I modified the contents of the structure that was previously created by MediaCodecService and added a subtitle_fd attribute, file descriptor to the decoded bitmap subtitle. When a message is received by the MediaServer's Nuplayer for rendering, the code tries to map the aforementioned file descriptor.

Unfortunately, the result of the call to ::mmap is always MAP_FAILED.

Do you have an idea of what I missed ?

Code of the MediaCodecService part

AVSubtitleRect *rect = sub->rects[0];
size_t len = sizeof(*rect);
int fd = ashmem_create_region("subtitle rect", len);
ashmem_set_prot_region(fd, PROT_READ | PROT_WRITE);
void* ptr = ::mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (ptr == MAP_FAILED) {
    ALOGI("%s[%d] dvb ptr == MAP_FAILED", __FUNCTION__, __LINE__);
} else {
    ALOGI("Success creating FD with value %d", fd);
}
memcpy(ptr, rect, len);
sub->subtitle_fd = fd;
sub->subtitle_size = len;

Code of the MediaServer part

int fd = mSubtitle->subtitle_fd;
size_t len = mSubtitle->subtitle_size;
ALOGI("Trying to map shared memory with FD = %d", fd);
void* ptr = ::mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (ptr == MAP_FAILED) {
    ALOGI("Subtitle mmap ptr==MAP_FAILED %s", strerror(errno));
} else {
    ALOGI("Subtitle get ptr %p", ptr);
}
AVSubtitleRect *rect = (AVSubtitleRect *)ptr;

Thank you so much !

Have you seen `ALOGI("Success creating FD with value %d", fd);`? Please add the `fd` to `ALOGI("Subtitle mmap ptr==MAP_FAILED %s", strerror(errno));`. — Gluttton, Aug 13 '18 at 07:53
Yes I have seen the ALOGI in LogCat. It displays a positive value every time (between 19 and 45 in my tests). When I log the FD value in the second code snippet, it is always the same value as the one created in the first code snippet. Unfortunately, the ::mmap function called with this FD does not succeed. — François.CA, Aug 13 '18 at 16:55
Have you tried to disable SELinux (start Android in permissive mode)? — Gluttton, Aug 13 '18 at 17:18
SELinux is disabled on my platform for now. `adb shell getenforce` returns **Permissive**. The same error happens whether SElinux is permissive or not. — François.CA, Aug 14 '18 at 21:12

Using Android Nougat's shared memory for media decoding

0 Answers0