The 65k reference limit of a DEX file is a problem that any developer might face sooner or later while working on an Android project and at some degree we had to deal with it at Sysmosoft as well.
In the community, it has been discussed many times how to deal with it so, in this article, after briefly explaining what we should do in such a situation, we’ll see how multidex actually works behind the scenes.
Any APK package has at least one DEX file and that’s where all the executable code of an application is stored. This includes not only our own implementation but also all the code of the libraries referenced, the dependencies declared in the
build.gradle file (or the
pom.xml file if we’re using Maven). So common libraries such as Android Support Libraries, Retrofit, ButterKnife, Guava, Jackson, Google Play Services, … if used by our application, will be taken into account while producing the DEX file.
The size of the DEX file’s method index is 16 bit, so it means that 65536 represents the total number of references that can be invoked by the code within a single DEX file. If building our application we surpass this limit, the dx tool throws an error, causing the failure of the APK packaging.
Error:Your app has more methods references than can fit in a single dex file. See https://developer.android.com/tools/building/multidex.html Error:Execution failed for task ':app:transformClassesWithDexForDebug'. > com.android.build.api.transform.TransformException: com.android.ide.common.process.ProcessException: org.gradle.process.internal.ExecException: Process 'command '/Library/Java/JavaVirtualMachines/jdk1.7.0_60.jdk/Contents/Home/bin/java'' finished with non-zero exit value 2
So, what should we do in this case? Actually we could think about it as a great occasion to review our code. In fact, before enabling the multidex support, two steps should be done.
First, review direct and transitive dependencies of our application: it is worth taking a bit of time and do this review, because we may find out we are including a very large library just to use a few of its methods. Anyway, don’t be surprised if everything is fine: if we did our job consciously while adding this dependency, we should have already picked the best option for our needs.
Also, if our application is a mature project, we could also realize that this large library is widely used in our code base: thus, a bit of time would be required to refactor our implementation, with the risk of introducing few regressions.
Anyway, one thing to double check very carefully are the transitive dependencies, because there’s the chance we could exclude few of them. A typical example is the Location API provided by Google: as of version
8.4.0, if we declare
play-services-location as a dependency, we get also
play-services-maps, but maybe we don’t need map support in our application
The second option we have is to shrink the code using Proguard: we could configure this tool to remove all the unused code, ensuring it’s not included in the final APK, reducing the number of referenced methods in our DEX file. We don’t need to optimize or obfuscate the code, simply to shrink it. The problems here could arise if we have never enabled Proguard at all in our project, because very likely we’ll face crashes at runtime due to missing classes removed by mistake. In this case, we need to add specific rules in the Proguard config file to explicitly tell the tool to keep those classes.
If neither of these two options is not enough for our project, so it doesn’t produce a DEX file with less than 65536 referenced methods, the way to go is to enable the multidex support.
At this point it should be clear that keeping an eye on the methods count is quite important, because dealing with code refactoring, Proguard or multidex support later in the life of an application could be tricky or anyway take away some time that could be spent implementing new features or improving the user experience.
There are many solutions to get this value, here I simply mention two of my favourites:
dex-methods-count (command-line tool)
dexcount-gradle-plugin (Gradle plugin, so it can be conveniently integrated in the application build process)
I’d like also to mention this online tool (they also provide a Gradle plugin): Methods count. That’s a good place to check a new dependency before integrating it in an application: the only limitation is that the library repository must be Maven Central, jCenter or JitPack.
The official documentation about multidex is particularly well written so there’s no point in doing a mere copy & paste here.
One important thing to highlight though, is about the minimum SDK version supported by the application: in fact, if it runs on Android 5.0 and higher, the configuration would be quite simple, because multiple DEX files in an APK are natively loaded by ART. For lower versions of Android, the multidex support library must be used.
That’s s an important point because I’ve seen a lot of questions on StackOverflow about the
java.lang.NoClassDefFoundError exception while running a multidex enabled application on Android versions prior to Lollipop.
Multidex support library
Let’s start enabling multidex using the support library, following the official instructions. Just two things to note: the version of the library used for this article is
1.0.1, while the application is the Empty Activity project created from Android Studio using the wizard.
Also, since there’s not much code implemented, to force the need to enable multidex, we should add a bunch of dependecies, like the following:
compile 'com.android.support:appcompat-v7:23.1.1' compile 'com.android.support:recyclerview-v7:23.1.1' compile 'com.android.support:multidex:1.0.1' compile 'com.squareup.picasso:picasso:2.5.2' compile 'com.squareup.retrofit:retrofit:2.0.0-beta2' compile 'com.facebook.fresco:fresco:0.8.1' compile 'com.google.guava:guava:18.0' compile 'com.google.code.gson:gson:2.5' compile 'com.android.support:design:23.1.1' compile 'com.facebook.android:facebook-android-sdk:4.8.2' compile 'com.github.bumptech.glide:glide:3.6.1' compile 'org.apache.commons:commons-lang3:3.4' compile 'com.google.android.gms:play-services:8.4.0'
Once the APK is produced, we can list its content using aapt to see that there are actually many DEX files in it.
You can also take advantage of one of the many tools available online to browse an APK and have a look at the methods inside each of the DEX file. One simple to install and use is ClassyShark: here is also available a quick introduction about inspecting multidex APK files.
The main aim of the library is to patch the application context class loader in order to load classes from more than one DEX file. The primary classes.dex must contain the classes necessary for calling this class methods, otherwise the application will crash. Secondary DEX files, named classes2.dex, classes3.dex, … found in the application APK will be added to the classloader after the first call to
install(Context) of the
The logic of interest is in the
installSecondaryDexes method of the
MultiDex class: first, secondary DEX files are retrieved and then
dalvik.system.DexPathList pathList field is modified to append additional DEX file entries. In this way, the
ClassLoader instance at
Application level (retrieved using
getClassLoader()) is also aware of the classes present in the additional DEX files.
Source code of the latest version of the multidex support library can be found here.
This operation is performed when the application is started: in fact, if the application already extends the
Application class, the requirement of the multidex support library is to call the
install method from
attachBaseContext (overriding it if necessary).
In short, when the application is started and the
Application instance is attached, the patched class loading system is run to load the additional DEX files present in the APK file.
Of course, this extra work, required at startup, has an impact on performances: that’s explained in details by this great article, which I suggest you to read.
Actually the support library is only one part of the whole multidex feature, because, as we have seen, it simply takes care of loading all the classes when we start our application and nothing much more. But at point, the APK already has multiple DEX files in it.
So who’s responsible for generating all the different DEX files?
The dx tool is the executable in charge of generating the DEX file (or more) to be included in the final APK package, which will be installed on the device.
dx in a terminal gives us the help output for the command and three options are related to multidex:
--multi-dex: allows to generate several DEX files if needed.
<file> is a list of class file names, classes defined by those class files are put in
--minimal-main-dex: only classes selected by
--main-dex-list are to be put in the main DEX.
The first one is obvious: it’s used when the build process has to support multidex, producing more than one DEX file. So, when we enable multidex, this options is passed to dx.
The second option is a path to a file where class file names are listed: these files will be put into the main DEX file. This is necessary because some classes cannot be stored in secondary DEX files, otherwise the application could crash when launched. If we use the multidex support library, a typical example of a file to be kept into the main DEX file, is the
Application class (the one provided by the library or our own implementation). As we have seen, this file is responsible for loading the classes of the secondary DEX files, and it’s executed at application startup time, so it must be in the main DEX file.
A typical auto-generated file will list at least the following classes using the multidex support library:
android/support/multidex/ZipUtil.class android/support/multidex/ZipUtil$CentralDirectory.class android/support/multidex/MultiDex$V14.class android/support/multidex/MultiDexExtractor$1.class android/support/multidex/MultiDexExtractor.class android/support/multidex/MultiDexApplication.class android/support/multidex/MultiDex.class android/support/multidex/MultiDex$V19.class android/support/multidex/MultiDex$V4.class
The third option is again quite simple to understand: if set, only the classes listed in
--main-dex-list will be put into the primary DEX file.
There’s also a fourth option, labelled as undocumented test option:
--set-max-idx-number. It sets the maximum number of method references to be put into the main DEX file: that’s it, we can instruct dx to put at least a specific number of method references in any DEX file to be produced.
dx source code is open, so let’s start having a look at the main class for the class file translator, specifically at the
runMultiDex() method. The first line of interest is:
That’s basically where the
classesInMainDex HashSet is filled with the class files to be kept in the primary DEX file.
Then, the second method to look to have a better understanding is
processAllFiles(), which is where, very unsurprisingly, all the DEX files are created and filled with all the specified classes. Here we can see how the tool tries to force the classes listed by
--main-dex-list, to be part of the main DEX file: looking at the
DexException that could be thrown, it’s clear that we don’t have to try to have too many classes as part of this list, otherwise we could face the same initial issue for which we have enabled the multidex support.
That’s all about multidex, it’s not worth diving into more specific details: it was just important to highlight what’s behind the scenes when we simply add
multiDexEnabled in our gradle script.