11

I'm trying to grab 784 MiB of memory. Yes, I know that is a lot for a 32-bit phone, but the following call worked before Android 5.0:

mmap(0, 0x31000000, PROT_NONE, MAP_ANON | MAP_SHARED, -1, 0);

However, on three different devices from different manufacturers, upgrading to Android 5.0 has broken this. I assume this is some change in memory allocation functionality in 5.0; maybe different flags need to be passed in?

Here's the error message returned in logcat:

E/libc﹕ mmap fail (pid 9994, tid 10125, size 822083584, flags 0x21, errno 12(Out of memory))
sigmabeta
  • 1,174
  • 1
  • 12
  • 28
  • The error message seems quite obvious; did you check that there is enough memory available? – Phillip Jun 08 '15 at 07:44
  • I do not know of a robust way to check exactly how much memory is free and accessible by native code. However, that's not really the part I don't understand - I don't understand why the same code would work on the same device running Android 4.4, and then fail on 5.0. It's unlikely Android's memory footprint increased that much - I feel it's more likely that some new compiler flag is required, or something like that. – sigmabeta Jun 08 '15 at 13:29
  • Just use `adb shell free -m` from a connected PC. If you don't have access to a device, SO has various solutions to how to query the available memory from C in linux. (You should *really* do that.) – Phillip Jun 08 '15 at 16:55
  • Yes, `free` and `vmstat` both report around 128MiB free on my Nexus 6 (32-bit). However, it reports approximately the same amount free on my Nexus 9 (64-bit), which completes the `mmap` call successfully. – sigmabeta Jun 08 '15 at 17:26
  • [Are you sure that you interpret the output correctly](http://serverfault.com/a/38074), and [do you then proceed to actually use all of the mapped memory on the N9](http://www.etalabs.net/overcommit.html)? – Phillip Jun 08 '15 at 18:22
  • The virtual memory limit was probably changed in 5.0? – Tavian Barnes Jun 12 '15 at 00:58
  • @TavianBarnes can you provide any more information about this? Nothing I've read indicates that there is any limit to native memory allocation imposed by the operating system. – sigmabeta Jun 12 '15 at 03:37
  • @signmabeta Can you please check more dmesg log? – Qylin Jun 12 '15 at 08:30

3 Answers3

10

At the point where the mmap() fails, open /proc/self/maps and copy the contents to a temp file, then examine the file in an editor. You should see a bunch of entries like this:

12e01000-42c00000 ---p 00201000 00:04 11639      /dev/ashmem/dalvik-main space (deleted)
55281000-5d500000 r--s 00000000 00:16 61         /storage/sdcard1/blah
5d500000-67e80000 rw-p 00000000 00:00 0          [anon:libc_malloc]
67ea4000-682cc000 r-xp 00000000 b3:17 114807     /system/vendor/lib/libsc-a3xx.so
682cc000-682f4000 r--p 00427000 b3:17 114807     /system/vendor/lib/libsc-a3xx.so

The numbers on the left are virtual address ranges (start / end) for the process. When you create a new mapping, it needs to fit in the gap between mappings.

In the example above, there's a nice big gap between the end of the first entry at 0x42c00000 and the start of the next at 0x55281000. That's about 294MB. There's no space between the next two, and only a small one after that.

If you look at your process map, and don't find a gap big enough to hold your file, you have your answer. The region between 0x00000000 and 0xbfffffff is generally available to 32-bit apps, but the app framework uses up a great deal of it. (The top 1GB is mapped to the kernel.)

My guess is that some combination of ASLR and changes to the way virtual memory is allocated in Lollipop have led to this problem. In the map attached to this similar question, the largest gap found was about 300MB. There are two large "dalvik" regions, one 768MB (at 12e01000), one 1.2GB (at 84d81000). (Since you are running Lollipop these are actually due to ART rather than Dalvik, but apparently the label stuck.)

One possibility is that ART has higher virtual memory requirements than Dalvik did, and the large allocations are making it difficult for applications to get large mapped regions. It's also possible that ART is over-allocating due to a bug. You may want to test on Marshmallow to see if something got fixed.

Whatever the case, you can't create a mapping if there isn't a contiguous virtual memory address region large enough to hold it. With the app framework usage and the enormous ART allocations seen in the other question, a 768MB mapping wouldn't be possible even if the virtual address space weren't fragmented. You will need to map smaller sections of the file, and possibly un-map them as you work to make room.

It might be worth filing a bug on b.android.com. Attach a copy of your process map file, and be specific about version of Android and device.

For a bit more about interpreting /proc/maps output, see e.g. this answer.

Community
  • 1
  • 1
fadden
  • 51,356
  • 5
  • 116
  • 166
  • This is an excellent answer. It will be some time before I get around to experimenting with any sort of solution to this problem (and in the mean time, the project in question is limited to 64-bit devices only), but this sounds like it answers at least the question of why an OS update broke the code in question. – sigmabeta Nov 30 '15 at 16:14
0
  • Recompile your application library to match lollipop and try again
  • Update your SDK
  • Be sure to set target platform to lollipop on your application settings
  • Reduce the allocated memory and check again
intika
  • 8,448
  • 5
  • 36
  • 55
0

Have you tried largeHeap option?

In very special situations, you can request a larger heap size by setting the largeHeap attribute to "true" in the manifest tag. If you do so, you can call getLargeMemoryClass() to get an estimate of the large heap size.

https://developer.android.com/training/articles/memory.html#CheckHowMuchMemory

More Details:

https://developer.android.com/guide/topics/manifest/application-element.html#largeHeap

mayo
  • 3,845
  • 1
  • 32
  • 42