10

Background

It seems some old Android OSs (and maybe even the newest ones) have a limitation on the amount of code each app can hold.

As I've found, the limitation is on a buffer called "LinearAlloc" .

On 2.2 or 2.3 it's about 5-8 MB , and I think it's 16 or more on others.

The problem

If you have a too large code (and apps can reach this state), you won't be able to install the app at all on older devices, getting the next error (also reported here) :

Installation error: INSTALL_FAILED_DEXOPT
Please check logcat output for more details.
Launch canceled!

What I've found

One solution is to just remove as much code and libraries as possible, but on some huge projects such a thing is very hard to do.

I've found the next links talking about how Facebook solved this, by somehow increasing the limit:

Also, Google has posted how to solve it by loading code dynamically :

http://android-developers.blogspot.co.il/2011/07/custom-class-loading-in-dalvik.html

The question

How did Facebook do it?

Is it possible to overcome this in other ways too?

Is there any free library that increases/removes the limitation of this buffer?

What is the limitation on newer Android versions, if there is any?

How do other huge apps (and games) handle this issue? Do they put their code into C/C++?

Would loading the dex files dynamically solve this?

Community
  • 1
  • 1
android developer
  • 114,585
  • 152
  • 739
  • 1,270
  • As I understand, the FB folks used the fact that Android 2.2 and 2.3 will not change anymore, so if you see 2.x, you can more or less safely assume that you know what source it was built from. As I understand, they used the JNIEnv* passes to each JNI function to find out the buffer address. You have to read the VM source to understand what they were doing. It is possible that you can do something by splitting your app in several processes (at the cost of IPC). At least, two processes should use two different VMs. But I myself did not try this for the code amount limitation problem. – 18446744073709551615 Feb 19 '14 at 11:44
  • @18446744073709551615 Yes they have many words there that I don't understand, and I assume they are from Android source code (probably native code too). However, what I think the articles say is that FB somehow managed to increase this limitation. So I wonder how they did it and if there is a simple way to do it that someone else has found. I also wonder if it's considered as a hack to change this value, and if it works on newer android versions. – android developer Feb 19 '14 at 12:07
  • _newer versions_ -- No, you have to hack (yes) each version separately. I guess they had no problem with the newer versions. As to "how" -- you find (the pointer to) that buffer in the source and learn how it is allocated; you find out how to find that pointer via JNIEnv*; you allocate the required amount of memory and (!!!) store the address into that pointer. Probably you can do a realloc() (an in-place resize would be ideal, if that does happen repeatably). And you have to learn how to deallocate the memory when the application terminated (OTOH, leaks get cleared when the process ends). – 18446744073709551615 Feb 19 '14 at 12:24
  • Do you say all that in a theoretical speaking , or did you really have something that does it? if you did it, can you please share your code/project? Also, why would there be leaks? those are libraries that the app uses... – android developer Feb 19 '14 at 12:50
  • 1
    It is a theoretical speaking from someone who ported a significant amount of pre-existing C/C++ code to Android. I myself did not hack into the VM, but I used separate processes in practice as a remedy for memory leaks in some really old C/C++ libraries. And why leaks -- if you substitute the system buffer with you own one, you are very close to a memory leak. (PS again about leaks: _why there would be no leaks? You really believe you have fixed the last bug?_) – 18446744073709551615 Feb 20 '14 at 05:25
  • I see, but I still can't see it as a memory leak. if you use the memory allocated for the libraries, it can't be considered memory leak. memory leak is when the app can't free some allocated memory even though it will never use what the memory was allocated for. it's a memory that will never be used and will never be freed from a specific time (for example if you keep a reference to the splash screen bitmap even after it was shown to the user). there is also the notion of wasted memory, where you use some memory for too rare times, but that's not a memory leak. – android developer Feb 20 '14 at 06:34
  • Ok, putting everything back in context: if you substitute the original VM buffer with your own one, you have to make sure that the original buffer gets finally freed, otherwise it will be a memory leak. OTOH, leaks get cleared when the process ends, so you probably don't need to care. On the 3rd hand, it is keeping the original buffer allocated during the whole process runtime that should be considered a leak. – 18446744073709551615 Feb 20 '14 at 10:57
  • @18446744073709551615 oh you are talking about the original buffer, not the one that has the libraries. now i see. but you can free it on the beginning, after you've created the new one, no? – android developer Feb 20 '14 at 11:02
  • This depends on what the old buffer already contains and how that stuff is referenced. In the best case it will be empty and you will be able to free it by the method that corresponds to the method of its allocation (and they are many, including malloc(), new something, new something[n], and os-specific methods.) – 18446744073709551615 Feb 20 '14 at 11:10
  • @18446744073709551615 I see. if you ever implement this hack, can you please share it here? – android developer Feb 20 '14 at 11:16
  • I do not think this will happen in the near future. The apps on which I work are not so big. – 18446744073709551615 Feb 20 '14 at 11:59
  • How about reducing the code disk size by converting your helper classes to JAR files adding to your project. I assume there will be a lot of self-written Helper classes, converting all of them to a jar file can reduce the size drastically. – Parth Kapoor Apr 14 '14 at 13:21
  • @Eu.Dr. converting to jar files doesn't help, as they are still code just like others. jar files are just zipped files which in this case contain code... it's also what happens when you use android-library projects. – android developer Apr 14 '14 at 22:48
  • @androiddeveloper Zippping files (JAR) will reduce their size, and this is what needs to be done here. Or do you mean the size reduction will not be so much significant? – Parth Kapoor Apr 15 '14 at 02:48
  • @Eu.Dr. it won't , since all source code is merged anyway and the APK packaging already compress everything (including both code and resources). besides, it's not the file size that matters. it's how much memory it takes when the whole app is loaded, but for some reason dalvik doesn't care if it's later. that's why there are solutions like this : http://android-developers.blogspot.co.il/2011/07/custom-class-loading-in-dalvik.html , which just tell you to load the extra code programmatically. the jar can be within the app and you could load it when you need it. – android developer Apr 15 '14 at 08:04
  • @Eu.Dr. I agree that it's confusing, and I don't even understand how come there is a limitation, and why it's such a low RAM limitation. I think it's still an issue even on the newest Android versions. – android developer Apr 15 '14 at 09:20

2 Answers2

1

The limit is the total number of method references:

A middle ground between doing nothing and the multi-dex approach described in the FB/Google articles is to use a tool like ProGuard to remove references to unused code at the Java level. See:

rtsai2000
  • 338
  • 1
  • 5
  • minimizing code is not a solution as it won't help you when you reach a certain amount of code, and it's already being used anyway... :( – android developer Apr 18 '14 at 20:26
0

There is a new solution, made by Google:

It seems all you have to do is any of the next things: - extend from "MultiDexApplication" instead of from "Application" - call MultiDex.install(context) in your application's attachBaseContext

But now I wonder:

  1. Is that really it?
  2. Does it have any issues ? Does it affect performance?
  3. How does it work?
  4. What should be done with ContentProvider, as it's getting called before Application gets initialized?
  5. The post says "gives you MultiDex support on all API 4+ devices (well, until v21, where you get this natively)" . Does it mean that from v21 it will be the default behavior, or just that the class will be built in and you won't need to use the support library's class ?
  6. Will this solution work on Eclipse too?
android developer
  • 114,585
  • 152
  • 739
  • 1,270
  • All the Android components (ContentProviders among them) should be placed in your main dex file. I wrote a post that summarizes (and hopefully answers your other questions) this topic: http://blog.osom.info/2014/10/multi-dex-to-rescue-from-infamous-65536.html – Alex Lipov Oct 27 '14 at 12:08
  • @AlexLipov Didn't read yet, but this looks like a lot of text and steps for something that was written so shortly by Google. Does it work? can it work even on Eclipse? – android developer Oct 27 '14 at 15:30
  • @android_developer from my experience it works well. maven-android-plugin has support for multi-dex. – Alex Lipov Oct 27 '14 at 19:22
  • @AlexLipov Thank you. Hope Google will introduce it officially and make it much easier to use. – android developer Oct 27 '14 at 22:56