91

I have a rather large Android app that relies on many library projects. The Android compiler has a limitation of 65536 methods per .dex file and I am surpassing that number.

There are basically two paths you can choose (at least that I know of) when you hit the method limit.

1) Shrink your code

2) Build multiple dex files (see this blog post)

I looked into both and tried to find out what was causing my method count to go so high. The Google Drive API takes the biggest chunk with the Guava dependency at over 12,000. Total libs for Drive API v2 reach over 23,000!

My question I guess is, what do you think I should do? Should I remove Google Drive integration as a feature of my app? Is there a way to shrink the API down (yes, I use proguard)? Should I go the multiple dex route (which looks rather painful, especially dealing with third party APIs)?

Jared Rummler
  • 37,824
  • 19
  • 133
  • 148
  • 2
    I happen to love your app. Have you thought about making a required download of all the extra libs in a pseudo `apk` form? I personally would like to see _Drive_ integration – JBirdVegas Mar 18 '13 at 07:44
  • Hmm, can you explain why proguard isn't solving the problem? – Kevin Bourrillion Mar 18 '13 at 14:14
  • Proguard helps but not by much it seems. To test, I created the project from the [quickstart guide](https://developers.google.com/drive/quickstart-android). I added the suggested config to proguard and the result saved me about 2-3k methods. That helps, but the project was still above 22,000 which is a big chunk. – Jared Rummler Mar 18 '13 at 15:37
  • 8
    Facebook recently documented their workaround for what seems like an almost identical problem in their Android app. May be useful: http://www.facebook.com/notes/facebook-engineering/under-the-hood-dalvik-patch-for-facebook-for-android/10151345597798920 – Reuben Scratton Mar 18 '13 at 17:49
  • 4
    Starting to head down the multiple dex route. I successfully created a secondary dex file to work with Google Drive. I feel bad for anyone that needs guava as a dependency. :P It's still a pretty big issue for me though – Jared Rummler Mar 18 '13 at 20:33
  • @JaredRummler Hi, I encounter a similar issues as you. However, I'm stuck during eclipse to ant transition. Do you mind to take a look? http://stackoverflow.com/questions/18629021/safe-way-to-migrate-from-existing-eclipse-built-android-project-to-ant-build – Cheok Yan Cheng Sep 05 '13 at 07:27
  • @JaredRummler if you are using actionbarsherlock, switch to ACtionBarAppCompat. – petey Dec 20 '13 at 15:31
  • What version of guava are you using now? – petey Dec 20 '13 at 16:47
  • 4
    how do you count the methods? – Bri6ko Jun 02 '14 at 22:48
  • @Bri6ko https://gist.github.com/JakeWharton/6002797 cat $dexfile | head -c 92 | tail -c 4 | hexdump -e '1/4 "%d\n"' – Jared Rummler Jun 06 '14 at 08:13
  • 1
    Some additional notes here: http://stackoverflow.com/questions/21490382/ (including a link to a utility that will list the method references in an APK). Note the 64K limit is unrelated to the Facebook issue linked a few comments up. – fadden Jun 09 '14 at 19:38
  • Here is a script I wrote for counting methods in a jar folder https://gist.github.com/toms972/c83504df2da1176a248a – Tom Susel Jul 28 '14 at 09:11
  • Just for any "greenhorns" out there who were born into the Mac OS X era. This 64K dex limit absolutely reminds me of the end of Mac OS 8/9 era: CodeWarrior's 64K TOC limit: http://compgroups.net/comp.mac.codewarrior/toc-size-to-big-how-to-fix/2946478 Not only global variables, but virtual functions in classes and other stuff increased the TOC size too, so if you simply had a big enough executable, you were screwed. A decade after that I encounter a 64K limit again. ART is replacing Dalvik, hopefully it won't have such limit. – Csaba Toth Jan 29 '15 at 03:08
  • 1
    @CsabaToth ART didn't change the limit. – Jared Rummler Jan 29 '15 at 03:42
  • @JaredRummler Nooo! BTW, not for nitpicking, but 65536 is 64K, not 65K. Although even Google refers to 65K https://developer.android.com/tools/building/multidex.html – Csaba Toth Jan 29 '15 at 05:36
  • 1
    We wrote a Gradle plugin that gives you the method count after every build @Bri6ko – philipp Jun 05 '15 at 00:01
  • @CsabaToth 65K refers to 65,000 – cren90 Jul 28 '15 at 22:54
  • It is 64k, not 65k. 65535. – user207421 Jul 21 '16 at 04:21

12 Answers12

69

It looks like Google has finally implementing a workaround/fix for surpassing the 65K method limit of dex files.

About the 65K Reference Limit

Android application (APK) files contain executable bytecode files in the form of Dalvik Executable (DEX) files, which contain the compiled code used to run your app. The Dalvik Executable specification limits the total number of methods that can be referenced within a single DEX file to 65,536, including Android framework methods, library methods, and methods in your own code. Getting past this limit requires that you configure your app build process to generate more than one DEX file, known as a multidex configuration.

Multidex support prior to Android 5.0

Versions of the platform prior to Android 5.0 use the Dalvik runtime for executing app code. By default, Dalvik limits apps to a single classes.dex bytecode file per APK. In order to get around this limitation, you can use the multidex support library, which becomes part of the primary DEX file of your app and then manages access to the additional DEX files and the code they contain.

Multidex support for Android 5.0 and higher

Android 5.0 and higher uses a runtime called ART which natively supports loading multiple dex files from application APK files. ART performs pre-compilation at application install time which scans for classes(..N).dex files and compiles them into a single .oat file for execution by the Android device. For more information on the Android 5.0 runtime, see Introducing ART.

See: Building Apps with Over 65K Methods


Multidex Support Library

This library provides support for building apps with multiple Dalvik Executable (DEX) files. Apps that reference more than 65536 methods are required to use multidex configurations. For more information about using multidex, see Building Apps with Over 65K Methods.

This library is located in the /extras/android/support/multidex/ directory after you download the Android Support Libraries. The library does not contain user interface resources. To include it in your application project, follow the instructions for Adding libraries without resources.

The Gradle build script dependency identifier for this library is as follows:

com.android.support:multidex:1.0.+ This dependency notation specifies the release version 1.0.0 or higher.


You should still avoid hitting the 65K method limit by actively using proguard and reviewing your dependencies.

Jared Rummler
  • 37,824
  • 19
  • 133
  • 148
53

you can use the multidex support library for that, To enable multidex

1) include it in dependencies:

dependencies {
  ...
  compile 'com.android.support:multidex:1.0.0'
}

2) Enable it in your app:

defaultConfig {
    ...
    minSdkVersion 14
    targetSdkVersion 21
    ....
    multiDexEnabled true
}

3) if you have a application class for your app then Override the attachBaseContext method like this:

package ....;
...
import android.support.multidex.MultiDex;

public class MyApplication extends Application {
  ....
   @Override
   protected void attachBaseContext(Context context) {
    super.attachBaseContext(context);
    MultiDex.install(this);
   }
}

4) if you don't have a application class for your application then register android.support.multidex.MultiDexApplication as your application in your manifest file. like this:

<application
    ...
    android:name="android.support.multidex.MultiDexApplication">
    ...
</application>

and it should work fine!

Prakhar
  • 2,270
  • 19
  • 26
32

Play Services 6.5+ helps: http://android-developers.blogspot.com/2014/12/google-play-services-and-dex-method.html

"Starting with version 6.5, of Google Play services, you’ll be able to pick from a number of individual APIs, and you can see"

...

"this will transitively include the ‘base’ libraries, which are used across all APIs."

This is good news, for a simple game for example you probably only need the base, games and maybe drive.

"The complete list of API names is below. More details can be found on the Android Developer site.:

  • com.google.android.gms:play-services-base:6.5.87
  • com.google.android.gms:play-services-ads:6.5.87
  • com.google.android.gms:play-services-appindexing:6.5.87
  • com.google.android.gms:play-services-maps:6.5.87
  • com.google.android.gms:play-services-location:6.5.87
  • com.google.android.gms:play-services-fitness:6.5.87
  • com.google.android.gms:play-services-panorama:6.5.87
  • com.google.android.gms:play-services-drive:6.5.87
  • com.google.android.gms:play-services-games:6.5.87
  • com.google.android.gms:play-services-wallet:6.5.87
  • com.google.android.gms:play-services-identity:6.5.87
  • com.google.android.gms:play-services-cast:6.5.87
  • com.google.android.gms:play-services-plus:6.5.87
  • com.google.android.gms:play-services-appstate:6.5.87
  • com.google.android.gms:play-services-wearable:6.5.87
  • com.google.android.gms:play-services-all-wear:6.5.87
Csaba Toth
  • 10,021
  • 5
  • 75
  • 121
  • Any information on how to do that within an Eclipse project? – Brian White Jan 26 '15 at 20:08
  • I'm not in a position to upgrade to that version yet. But if your project is Maven based, then hopefully you just have to solve that in you maven pom. – Csaba Toth Jan 27 '15 at 08:42
  • @webo80 Well, this only helps if you are up to 6.5.87 version. I wonder about petey's answer, that proguard strips down unused functions. I wonder if that involves 2rd party libs too, or just your own stuff. I should read more about proguard. – Csaba Toth Jan 29 '15 at 02:49
  • @BrianWhite The only solution for now seems to strip the .jar file with some external tool.. – milosmns Apr 24 '15 at 12:47
  • I ended up using this tool: https://gist.github.com/dextorer/a32cad7819b7f272239b – Brian White Apr 24 '15 at 13:16
  • 1
    @CsabaToth, You saved me! I added only a few of the list above instead of full 'com.google.android.gms:play-services', and that made the difference! – EZDsIt Apr 02 '16 at 20:15
9

In versions of Google Play services prior to 6.5, you had to compile the entire package of APIs into your app. In some cases, doing so made it more difficult to keep the number of methods in your app (including framework APIs, library methods, and your own code) under the 65,536 limit.

From version 6.5, you can instead selectively compile Google Play service APIs into your app. For example, to include only the Google Fit and Android Wear APIs, replace the following line in your build.gradle file:

compile 'com.google.android.gms:play-services:6.5.87'

with these lines:

compile 'com.google.android.gms:play-services-fitness:6.5.87'
compile 'com.google.android.gms:play-services-wearable:6.5.87'

for more reference, you can click here

akshay
  • 5,811
  • 5
  • 39
  • 58
7

Use proguard to lighten your apk as methods that are unused will not be in your final build. Double check you have following in your proguard config file to use proguard with guava (my apologies if you already have this, it wasn't known at time of writing) :

# Guava exclusions (http://code.google.com/p/guava-libraries/wiki/UsingProGuardWithGuava)
-dontwarn sun.misc.Unsafe
-dontwarn com.google.common.collect.MinMaxPriorityQueue
-keepclasseswithmembers public class * {
    public static void main(java.lang.String[]);
} 

# Guava depends on the annotation and inject packages for its annotations, keep them both
-keep public class javax.annotation.**
-keep public class javax.inject.**

In addition, if you are using ActionbarSherlock, switching to the v7 appcompat support library will also reduce your method count by a lot (based on personal experience). Instructions are located :

petey
  • 16,914
  • 6
  • 65
  • 97
  • this looks promissing but I got `Warning: butterknife.internal.ButterKnifeProcessor: can't find superclass or interface javax.annotation.processing.AbstractProcessor` when running `./gradlew :myapp:proguardDevDebug` – ericn Jan 05 '15 at 07:12
  • 1
    During development, though, proguard isn't generally run (at last not with Eclipse) so you can't benefit from the shrinkage until doing a release build. – Brian White Apr 24 '15 at 13:23
7

You could use Jar Jar Links to shrink huge external libraries like Google Play Services (16K methods!)

In your case you will just rip everything from Google Play Services jar except common internal and drive sub-packages.

pixel
  • 24,905
  • 36
  • 149
  • 251
4

For Eclipse users not using Gradle, there are tools that will break down the Google Play Services jar and rebuild it with only the parts you want.

I use strip_play_services.sh by dextorer.

It can be difficult to know exactly which services to include because there are some internal dependencies but you can start small and add to the configuration if it turns out that needed things are missing.

Brian White
  • 8,332
  • 2
  • 43
  • 67
3

I think that in the long run breaking your app in multiple dex would be the best way.

prmottajr
  • 1,816
  • 1
  • 13
  • 22
2

Multi-dex support is going to be the official solution for this issue. See my answer here for the details.

Community
  • 1
  • 1
Alex Lipov
  • 13,503
  • 5
  • 64
  • 87
2

If not to use multidex which making build process very slow. You can do the following. As yahska mentioned use specific google play service library. For most cases only this is needed.

compile 'com.google.android.gms:play-services-base:6.5.+'

Here is all available packages Selectively compiling APIs into your executable

If this will be not enough you can use gradle script. Put this code in file 'strip_play_services.gradle'

def toCamelCase(String string) {
String result = ""
string.findAll("[^\\W]+") { String word ->
    result += word.capitalize()
}
return result
}

afterEvaluate { project ->
Configuration runtimeConfiguration = project.configurations.getByName('compile')
println runtimeConfiguration
ResolutionResult resolution = runtimeConfiguration.incoming.resolutionResult
// Forces resolve of configuration
ModuleVersionIdentifier module = resolution.getAllComponents().find {
    it.moduleVersion.name.equals("play-services")
}.moduleVersion


def playServicesLibName = toCamelCase("${module.group} ${module.name} ${module.version}")
String prepareTaskName = "prepare${playServicesLibName}Library"
File playServiceRootFolder = project.tasks.find { it.name.equals(prepareTaskName) }.explodedDir


def tmpDir = new File(project.buildDir, 'intermediates/tmp')
tmpDir.mkdirs()
def libFile = new File(tmpDir, "${playServicesLibName}.marker")

def strippedClassFileName = "${playServicesLibName}.jar"
def classesStrippedJar = new File(tmpDir, strippedClassFileName)

def packageToExclude = ["com/google/ads/**",
                        "com/google/android/gms/actions/**",
                        "com/google/android/gms/ads/**",
                        // "com/google/android/gms/analytics/**",
                        "com/google/android/gms/appindexing/**",
                        "com/google/android/gms/appstate/**",
                        "com/google/android/gms/auth/**",
                        "com/google/android/gms/cast/**",
                        "com/google/android/gms/drive/**",
                        "com/google/android/gms/fitness/**",
                        "com/google/android/gms/games/**",
                        "com/google/android/gms/gcm/**",
                        "com/google/android/gms/identity/**",
                        "com/google/android/gms/location/**",
                        "com/google/android/gms/maps/**",
                        "com/google/android/gms/panorama/**",
                        "com/google/android/gms/plus/**",
                        "com/google/android/gms/security/**",
                        "com/google/android/gms/tagmanager/**",
                        "com/google/android/gms/wallet/**",
                        "com/google/android/gms/wearable/**"]

Task stripPlayServices = project.tasks.create(name: 'stripPlayServices', group: "Strip") {
    inputs.files new File(playServiceRootFolder, "classes.jar")
    outputs.dir playServiceRootFolder
    description 'Strip useless packages from Google Play Services library to avoid reaching dex limit'

    doLast {
        def packageExcludesAsString = packageToExclude.join(",")
        if (libFile.exists()
                && libFile.text == packageExcludesAsString
                && classesStrippedJar.exists()) {
            println "Play services already stripped"
            copy {
                from(file(classesStrippedJar))
                into(file(playServiceRootFolder))
                rename { fileName ->
                    fileName = "classes.jar"
                }
            }
        } else {
            copy {
                from(file(new File(playServiceRootFolder, "classes.jar")))
                into(file(playServiceRootFolder))
                rename { fileName ->
                    fileName = "classes_orig.jar"
                }
            }
            tasks.create(name: "stripPlayServices" + module.version, type: Jar) {
                destinationDir = playServiceRootFolder
                archiveName = "classes.jar"
                from(zipTree(new File(playServiceRootFolder, "classes_orig.jar"))) {
                    exclude packageToExclude
                }
            }.execute()
            delete file(new File(playServiceRootFolder, "classes_orig.jar"))
            copy {
                from(file(new File(playServiceRootFolder, "classes.jar")))
                into(file(tmpDir))
                rename { fileName ->
                    fileName = strippedClassFileName
                }
            }
            libFile.text = packageExcludesAsString
        }
    }
}

project.tasks.findAll {
    it.name.startsWith('prepare') && it.name.endsWith('Dependencies')
}.each { Task task ->
    task.dependsOn stripPlayServices
}
project.tasks.findAll { it.name.contains(prepareTaskName) }.each { Task task ->
    stripPlayServices.mustRunAfter task
}

}

Then apply this script in your build.gradle, like this

apply plugin: 'com.android.application'
apply from: 'strip_play_services.gradle'
Community
  • 1
  • 1
Roman Nazarevych
  • 7,513
  • 4
  • 62
  • 67
1

If using Google Play Services, you may know that it adds 20k+ methods. As already mentioned, Android Studio has the option for modular inclusion of specific services, but users stuck with Eclipse have to take modularisation into their own hands :(

Fortunately there's a shell script that makes the job fairly easy. Just extract to the google play services jar directory, edit the supplied .conf file as needed and execute the shell script.

An example of its use is here.

Tom
  • 6,946
  • 2
  • 47
  • 63
1

If using Google Play Services, you may know that it adds 20k+ methods. As already mentioned, Android Studio has the option for modular inclusion of specific services, but users stuck with Eclipse have to take modularisation into their own hands :(

Fortunately there's a shell script that makes the job fairly easy. Just extract to the google play services jar directory, edit the supplied .conf file as needed and execute the shell script.

An example of its use is here.

Just like he said, I replaces compile 'com.google.android.gms:play-services:9.0.0' just with the libraries that I needed and it worked.

Community
  • 1
  • 1
Tamir Gilany
  • 441
  • 7
  • 12