24

I am trying to integrate my application with Box, Dropbox, and Google Drive. All 3 of these services require a number of 3rd party jars. Additionally, my application already requires a few 3rd party jars. Now when I try to run my application from eclipse I get the following error:

Unable to execute dex: method ID not in [0, 0xffff]: 65536 Conversion to Dalvik format failed: Unable to execute dex: method ID not in [0, 0xffff]: 65536

It seems that this error occurs because my application has too many methods. I'm fairly certain the bulk of these methods are from the 3rd party jars, so it is unrealistic to try to solve this by simplifying my code. I found these two suggestions online.

  1. add dex.force.jumbo=trueto project.properties (and use adt version 21). I did this but still get the error.

  2. Use multiple dex files as explained here: http://android-developers.blogspot.co.il/2011/07/custom-class-loading-in-dalvik.html. This seems likely to be the only option, but I don't understand how it applies in my case. The issue is that services like Drive have too many dependencies. Wouldn't this solution require that I modify the Drive source to use inflection when referring to its dependencies? (this is clearly not an option).

  3. Use proguard to shrink remove unused code/methods. Exporting my application with proguard does work, and the document service integration works as expected on a >4.0 device. However, classnotfound errors are thrown when testing on a 2.3 device.

So, I am hoping for some advice on this issue. Is option 2 a solution for my case? Is there another solution I should consider?

danfuzz
  • 4,253
  • 24
  • 34
ab11
  • 19,770
  • 42
  • 120
  • 207
  • "All 3 of these services require a number of 3rd party jars" -- AFAIK, none of those require *any* third party JARs. You have *elected* to *use* JARs, and hence their dependent JARs, to access those services. However, since all of those services are accessible by means other than Java-based clients, they all have some underlying Web service API. For example, in the case of Dropbox, it is a REST-style API: https://www.dropbox.com/developers/core/api – CommonsWare Mar 19 '13 at 19:19
  • It would be more accurate to say that the "android libraries" for these services require a number of 3rd party jars? – ab11 Mar 19 '13 at 19:29
  • 1
    My point is that you do not need to use those libraries. Python developers do not use those JARs. Ruby developers do not use those JARs. JavaScript developers do not use those JARs. Those JARs are there for you as a convenience, but if they are giving you grief, drop down a layer and access their Web service API more directly. – CommonsWare Mar 19 '13 at 19:40
  • i understand your point and agree it is a valid, though far from ideal, solution. – ab11 Mar 19 '13 at 20:02
  • 1
    I would say proguard is your best bet. I would spend some time investigating why you're getting ClassNotFound exceptions. You probably need to tweak the proguard configuration. – JesusFreke Mar 19 '13 at 22:38
  • @JesusFreke One concern with the proguard solution: If proguard reduces the method count sufficiently, how can I check by how much? If its just barely small enough to avoid this issue, then I will be likely to have the same trouble in the future? – ab11 Mar 20 '13 at 14:21
  • @ab11 You can run dexdump on it with the -f option, and look at the method_ids_size field. – JesusFreke Mar 20 '13 at 22:07
  • @JesusFreke Awesome, thanks. I've looking for a way to do this. So, I install dexdump and run DexDump -f /data/app/com.myapp.myapp.apk. It spits out a bunch of information for each class in my app. Any suggestion for a way to use this to find the total method declarations? – ab11 Mar 21 '13 at 17:01
  • 2
    It's right near the top -- should look like `method_ids_size : 36700`. `dexdump` is part of the SDK. – fadden Mar 21 '13 at 21:27

8 Answers8

14

You can also develop one or more of these as a plugin to your main app, in the form of a separate APK available for download. That APK would expose some component that the main app would use -- since I do not know the nature of your integration with these services, I cannot make a more specific recommendation about that. You would use your own signature-level custom <permission> to secure communications between the two apps. And, as a bonus, if using the third-party library adds requirements for additional permissions, you would only need those permissions in the plugin APK, keeping your main APK smaller.

CommonsWare
  • 986,068
  • 189
  • 2,389
  • 2,491
  • this is the most realistic solution I've seen so far. I will leave the question unanswered for a bit, in hopes of more feedback, but I likely will end up using this. – ab11 Mar 19 '13 at 20:13
  • I am unable to find an example of this plugin architecture. I would like to find examples showing how to check if the plugin is installed, and how to use custom permissions for secure communications. Could you point me in the right direction? – ab11 Mar 19 '13 at 21:46
  • @ab11: Well, since the nature of a plugin architecture is heavily dependent upon what the plugins are actually doing, you will find it difficult to find general-purpose examples. Here is an example of a pair of projects implementing a plug-in UI by means of passing `RemoteViews` from the plugin to the host: https://github.com/commonsguy/cw-omnibus/tree/master/RemoteViews Roman Nurik's DashClock shows an app widget/lockscreen widget that supports plugins: https://code.google.com/p/dashclock/ – CommonsWare Mar 19 '13 at 21:49
7

***NEW**** All of the other answers are now outdated. Here's the new fix

Android 5.0 and higher

Multi-dex support is included automatically. From the docs:

Android 5.0 and higher uses a runtime called ART which natively supports loading multiple dex files from application APK files. ART performs pre-compilation at application install time which scans for classes(..N).dex files and compiles them into a single .oat file for execution by the Android device. For more information on the Android 5.0 runtime, see Introducing ART.

Below Android 5.0

Simply add the Android mult-dex support tool to your gradle build:

android {
    compileSdkVersion 21
    buildToolsVersion "21.1.0"

    defaultConfig {
        ...
        minSdkVersion 14
        targetSdkVersion 21
        ...

        // Enabling multidex support.
        multiDexEnabled true
    }
    ...
}

dependencies {
  compile 'com.android.support:multidex:1.0.0'
}
Jason
  • 13,563
  • 15
  • 74
  • 125
6

The Dalvik VM can have a maximum of 65536 methods per dex file, due to the bytecode instruction set not having a way to refer to method numbers requiring more than 16 bits (as pointed out by @danfuzz in the comments).

While it is possible to fix this using multiple dex files, Facebook found another fix that they could deploy within their app to get around the problem.

Raghav Sood
  • 81,899
  • 22
  • 187
  • 195
  • after many reads, I'm still not clear how I can implement their solution. I add something to my app to change an internal dalvik buffer? – ab11 Mar 19 '13 at 19:32
  • @ab11 You need to make a native extension using the android-ndk, which increases the size of the native buffer. – Raghav Sood Mar 19 '13 at 19:32
  • any possibility there is an example online of how to do this? also, in general, do you think this is a realistic option for a developer (me) that is very unclear about the solution? – ab11 Mar 19 '13 at 19:36
  • @ab11 AFAIK, facebook hasn't released the code for this (yet). Though you could probably be fine with splitting into multiple dex files, if your app isn't too complicated. – Raghav Sood Mar 19 '13 at 19:37
  • my concern with splitting into multiple dex files is that it would not be of use for the external services. my understanding is that once spilt, I can dynamically load and reflectively reference the methods in the second dex file. which i could use in my internal source, but would be of no use for the drive/box/dropbox code which contains the bulk of the methods. although, i'm not very clear on this and could definitely use feedback – ab11 Mar 19 '13 at 19:59
  • @ab11 I'm afraid I can't be of much practical help here. I know the theory, but I've never run into this problem myself, so can't say. – Raghav Sood Mar 19 '13 at 20:06
  • 5
    Note: The restriction isn't a "memory allocation issue." It is that the bytecode instruction set doesn't have a way to refer to method numbers requiring more than 16 bits. I had a fix for that problem in progress when I left the Android team, and it seems that nobody picked it up in the mean time (alas). – danfuzz Mar 19 '13 at 20:14
  • 1
    The Facebook issue *was* a memory allocation issue, and isn't the same as the 16-bit method reference issue. Note in particular their fix was only needed for older versions of Android (froyo / gingerbread) and does nothing on honeycomb and later. The 16-bit limit applies to all past and present versions equally. – fadden Mar 21 '13 at 21:26
  • Having this same issue, but I'm a bit confused as to why I'm getting it. I used to compile via ANT in Android Studio, but today I migrated it over to gradle manually (created a new project and copied the old code, modules, and support libs in). It's the same project as before, but obviously something changed be it the way the libraries are referenced or how gradle works itself which has caused it to start throwing this error. Any ideas? – JMRboosties May 15 '14 at 19:42
5

See vm/LinearAlloc.c and you can find this code: (5MiB under Android 2.3.3, 8MiB after Android 4.0 as my investigation)

#define DEFAULT_MAX_LENGTH (5*1024*1024)

...

LinearAllocHdr* pHdr;

...

pHdr->mapLength = DEFAULT_MAX_LENGTH;

I suppose that the 'Facebook fix' is editing this memory by using native C pointer. IMHO LinearAlloc problem and this method ID problem is different thing.

Francesco Jo
  • 63
  • 1
  • 4
3

I faced this issue recently. After scouring the web for some more detailed implementation, I realized there wasn't much out there other than:

I realized that the problem was not necessarily that there were too many methods in my code, but that the full dex of my code and other libraries was the issue. So, if I could compile my code against the libraries but not include them in the classes.dex, and then dex the libraries separately, and then put it all together at runtime it should work. The only issue left to address is the class loader, which Facebook mentioned in passing.

So with a little reflection and some Groovy code, I think I came up with a relatively stable way to package the libraries and the application code into separate dex files.

https://gist.github.com/nickcaballero/7045993

Nick Caballero
  • 944
  • 1
  • 8
  • 19
  • I'll look at that, but just wanted to inform you of my terrible solution. I have some external libraries that i only use for certain things; in deveolpment i don't need to include them in my application unless i'm working on those specific functionalities. so, i remove them from the libs folder so they aren't built into the app and put them on the class path so my code compiles; obviously my app will crash if those features are used, but i'll put the needed one back in to develop on it. When I build for release, I include the libs because progruard removes unused classes from the libs. – ab11 Oct 25 '13 at 14:52
1

Most of the problems with hitting the 65k method limit are related with the use of the mastodontic Google Play Services in your apps. Recently, you can get more granularity when using it.

Following this guide, you can use only parts that you want. Probably this will fix the problem, avoiding some black-magic tricks, or using multiDex. For example, if you only want Google Maps in your app (and you aren't using ads, wallet, google wear, analytics, etc...), using the entire dependency is a waste of time/space. You can use it that way:

compile com.google.android.gms:play-services-base:6.5.87
compile com.google.android.gms:play-services-maps:6.5.87

You can read the entire list of "parts" in this link

webo80
  • 3,365
  • 5
  • 35
  • 52
0

Here is a script I wrote for counting the number of methods in each jar (and in total) for a specific folder.

Once you count the methods you can concentrate on refactoring and removing heavy libraries.

Tom Susel
  • 3,397
  • 1
  • 24
  • 25
0

You need to enable the Dex support for that. So you need to do these steps:

  1. Gradle plugin v0.14.0 for Android adds support for multi-dex. To enable, you just have to declare it in build.gradle:
android {
   defaultConfig {
      ...
      multiDexEnabled = true
   }
}
  1. if app support > 5.0 (that is, if your minSdkVersion is 20 or below) you also have to dynamically patch the application ClassLoader, so it will be able to load classes from secondary dexes. for that you can add this lib.
 dependencies {
      ...
      compile 'com.android.support:multidex:1.0.0'
    }
  1. enable in code for that you have these option. choose one which suits you best

A. Add MultiDexApplication in manifest manifest 

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.android.multidex.App"> <application
android:name="android.support.multidex.MultiDexApplication">
</application>
</manifest>

B. Extend the application by MultiDexApplication

public class App extends MultiDexApplication { .. }

C. install it in application in attaching base context.

public class App {
    protected void attachBaseContext(Context base) {
        super.attachBaseContext(base);
        MultiDex.install(this);
        ..
    }

}

For more go through this link MultiDex.

RATHI
  • 5,129
  • 8
  • 39
  • 48