DuplicateFileException error while building Android APK

Question

I am using Android Studio 2.0 on Windows 7. When building my Android project, I get the following error:

:app:transformResourcesWithMergeJavaResForDebug FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:transformResourcesWithMergeJavaResForDebug'.
> com.android.build.api.transform.TransformException: com.android.builder.packaging.DuplicateFileException: Duplicate files copied in APK com/google/appengine/repackaged/org/apache/commons/codec/language/bm/sep_approx_spanish.txt
File1: C:\Users\USER\.gradle\caches\modules-2\files-2.1\com.google.appengine\appengine-api-1.0-sdk\1.9.28\e92c18272b555027d9ec666e7a89162f10638314\appengine-api-1.0-sdk-1.9.28.jar
File2: C:\Users\USER\.gradle\caches\modules-2\files-2.1\com.google.appengine\appengine-endpoints\1.9.28\bf2e8a74bd28e388b3487fc78a0c7adfa592fd5d\appengine-endpoints-1.9.28.jar`

I have been digging around for hours and can't see anything that causes this. I've deleted .gradle in my home directory, but it just rebuilds it all the same. Does anyone know what causes this?

Full gradle output:

Executing tasks: [:app:clean, :app:generateDebugSources, :app:mockableAndroidJar, :app:prepareDebugUnitTestDependencies, :app:generateDebugAndroidTestSources, :app:assembleDebug]

Configuration on demand is an incubating feature.
WARNING: Dependency org.apache.httpcomponents:httpclient:4.0.1 is ignored for debug as it may be conflicting with the internal version provided by Android.
     In case of problem, please repackage it with jarjar to change the class packages
WARNING: Dependency org.apache.httpcomponents:httpclient:4.0.1 is ignored for release as it may be conflicting with the internal version provided by Android.
     In case of problem, please repackage it with jarjar to change the class packages
Incremental java compilation is an incubating feature.
:app:clean
:app:preBuild UP-TO-DATE
:app:preDebugBuild UP-TO-DATE
:app:checkDebugManifest
:app:preReleaseBuild UP-TO-DATE
:backend:appengineDownloadSdk
:backend:compileJava
:backend:processResources UP-TO-DATE
:backend:classes
:backend:appengineEndpointsGetClientLibs UP-TO-DATE
:backend:appengineEndpointsExpandClientLibs UP-TO-DATE
:backend:compileEndpointsSrcJava
:backend:processEndpointsSrcResources UP-TO-DATE
:backend:endpointsSrcClasses
:backend:_appengineEndpointsAndroidArtifact UP-TO-DATE
:backend:jar UP-TO-DATE
:app:prepareComAndroidSupportAnimatedVectorDrawable2330Library
:app:prepareComAndroidSupportAppcompatV72330Library
:app:prepareComAndroidSupportSupportV42330Library
:app:prepareComAndroidSupportSupportVectorDrawable2330Library
:app:prepareComGoogleAndroidGmsPlayServicesAuth840Library
:app:prepareComGoogleAndroidGmsPlayServicesBase840Library
:app:prepareComGoogleAndroidGmsPlayServicesBasement840Library
:app:prepareComGoogleAndroidGmsPlayServicesMeasurement840Library
:app:prepareDebugDependencies
:app:compileDebugAidl
:app:compileDebugRenderscript
:app:generateDebugBuildConfig
:app:generateDebugAssets UP-TO-DATE
:app:mergeDebugAssets
:app:generateDebugResValues UP-TO-DATE
:app:processDebugGoogleServices
:app:generateDebugResources
:app:mergeDebugResources
:app:processDebugManifest
:app:processDebugResources
:app:generateDebugSources
:app:mockableAndroidJar UP-TO-DATE
:app:preDebugUnitTestBuild UP-TO-DATE
:app:prepareDebugUnitTestDependencies
:app:preDebugAndroidTestBuild UP-TO-DATE
:app:prepareDebugAndroidTestDependencies
:app:compileDebugAndroidTestAidl
:app:processDebugAndroidTestManifest
:app:compileDebugAndroidTestRenderscript
:app:generateDebugAndroidTestBuildConfig
:app:generateDebugAndroidTestAssets UP-TO-DATE
:app:mergeDebugAndroidTestAssets
:app:generateDebugAndroidTestResValues UP-TO-DATE
:app:generateDebugAndroidTestResources
:app:mergeDebugAndroidTestResources
:app:processDebugAndroidTestResources
:app:generateDebugAndroidTestSources
:app:compileDebugJavaWithJavac
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:app:compileDebugNdk UP-TO-DATE
:app:compileDebugSources
:app:buildInfoDebugLoader
:app:transformClassesWithExtractJarsForDebug
:app:transformClassesWithInstantRunVerifierForDebug
:app:transformClassesWithJavaResourcesVerifierForDebug UP-TO-DATE
:app:mergeDebugJniLibFolders
:app:transformNative_libsWithMergeJniLibsForDebug
:app:processDebugJavaRes UP-TO-DATE
:app:transformResourcesWithMergeJavaResForDebug FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:transformResourcesWithMergeJavaResForDebug'.
> com.android.build.api.transform.TransformException:     com.android.builder.packaging.DuplicateFileException: Duplicate files copied in APK com/google/appengine/repackaged/org/apache/commons/codec/language/bm/sep_approx_spanish.txt
File1: C:\Users\USER\.gradle\caches\modules-2\files-2.1\com.google.appengine\appengine-api-1.0-sdk\1.9.28\e92c18272b555027d9ec666e7a89162f10638314\appengine-api-1.0-sdk-1.9.28.jar
File2: C:\Users\USER\.gradle\caches\modules-2\files-2.1\com.google.appengine\appengine-endpoints\1.9.28\bf2e8a74bd28e388b3487fc78a0c7adfa592fd5d\appengine-endpoints-1.9.28.jar


* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 15.364 secs

From what I can tell, it's a conflict between two dependencies:

dependencies {
  appengineSdk 'com.google.appengine:appengine-java-sdk:1.9.28'
  compile 'com.google.appengine:appengine-endpoints:1.9.28'
  compile 'com.google.appengine:appengine-endpoints-deps:1.9.28'
  compile 'javax.servlet:servlet-api:2.5'
  compile 'com.google.http-client:google-http-client-jackson2:1.20.0'
  compile 'com.google.http-client:google-http-client-gson:1.19.0'
}

I think it's these two dependencies:

appengineSdk 'com.google.appengine:appengine-java-sdk:1.9.28'
compile 'com.google.appengine:appengine-endpoints:1.9.28'
compile 'com.google.appengine:appengine-endpoints-deps:1.9.28'

I'm not able to remove any of these, as it causes compile errors on the backend module. It looks like those two dependencies are almost identical, however. Should they both be in there? Any advice is appreciated!

Please post the entire Gradle Console output, not just these lines. See [this Stack Overflow question and answer](https://stackoverflow.com/questions/36698816/gradle-what-is-a-non-zero-exit-value-and-how-do-i-fix-it#comment60986098_36698816) for more. — CommonsWare, Apr 18 '16 at 17:11
OK, gradle output added. I have looked through the output, but none of it suggests anything to me. — James Funk, Apr 18 '16 at 17:18

score 7 · Accepted Answer · edited Jul 13 '16 at 04:55

7

To complete @dsh's answer:

In your dependency tree there are 2 jar files containing the same file named sep_approx_spanish.txt , this is not allowed.

To resolve your problem you have to track down which of your gradle dependencies has the two jars named appengine-api-1.0-sdk-1.9.28.jar and appengine-endpoints-1.9.28.jar; to do this you can open android studio's terminal console and type in:

//if ou are on windows
gradlew dependencies

//if you are on unix based os
./gradlew dependencies

You will see gradle doing a lot of things, but at a certain point it will print the entire dependency tree. Once you find which are the 2 dependencies are causing the problem, add this piece of code at the end of one of the dependency declarations

{
    exclude "sep_approx_spanish.txt"
}

So for example if the two dependencies are these

compile 'com.google.appengine:appengine-endpoints:1.9.28'
compile 'com.google.appengine:appengine-endpoints-deps:1.9.28

if must become

compile 'com.google.appengine:appengine-endpoints:1.9.28'{
        exclude "sep_approx_spanish.txt"
    }
 compile 'com.google.appengine:appengine-endpoints-deps:1.9.28

or

compile 'com.google.appengine:appengine-endpoints:1.9.28'
compile 'com.google.appengine:appengine-endpoints-deps:1.9.28'{
        exclude "sep_approx_spanish.txt"
    }

in this way you tell to gradle to not take into consideration that file for one of the two libraries, avoiding the file's duplication inside the apk.

edited Jul 13 '16 at 04:55

Mukesh Ram

6,248
4
19
37

answered Apr 18 '16 at 19:33

Apperside

3,542
2
38
65

Perfect answer, this solved it. I had to go through several txt files to get it to work, but once i had gotten them all sorted out, it was fine. Also, excellent work writing a very thorough answer :) – James Funk Apr 19 '16 at 00:17
glad to know it helped you :) – Apperside Apr 19 '16 at 08:49
1

@JamesFunk did this work for you? I ended up getting another error: Error:(26, 0) Gradle DSL method not found: 'com.google.appengine:appengine-endpoints:1.9.28()' – sublimental Aug 11 '16 at 21:30
1

@Mukesh I am getting gradle build error Gradle DSL method not found if i use compile ('com.google.appengine:appengine-endpoints-deps:1.9.28'){ exclude "sep_approx_spanish.txt" } compile 'com.google.appengine:appengine-endpoints:1.9.28' or compile ('com.google.appengine:appengine-endpoints-deps:1.9.28'{ exclude "sep_approx_spanish.txt" } compile 'com.google.appengine:appengine-endpoints:1.9.28' – user3024215 Aug 28 '16 at 06:21
1

@user3024215 the problem seem to be related to the exlude statment, without that it compiles. if you want to exclude a certain file you can try to put it in packagingOptions. To to that you should add something like packagingOptions { exclude 'META-INF/maven/com.google.guava/guava/pom.xml'} INSIDE THE android{} SECTION OF build.GRADLE, NOT INSIDE THE compile{} SECTION. obviously you must replace 'META-INF/maven/com.google.guava/guava/pom.xml' with 'META-INF/package/of/the/file/filename.extension'. I used this solution to eclude the file pom.xml in guava library – Apperside Aug 29 '16 at 14:01

score 2 · Answer 2 · answered Apr 18 '16 at 17:16

2

The error tells you that building your APK would result in two files named com/google/appengine/repackaged/org/apache/commons/codec/language/bm/sep_approx_spanish.txt. This is a failure because the APK can only contain one file at any given path. This is probably caused by two different jar files in your classpath that contain the file. In my project, this only occurred with files that were not actually used at runtime so I excluded them from the APK entirely.

answered Apr 18 '16 at 17:16

dsh

12,037
3
33
51

Although this technically is correct, I marked the other one as answer because it explained how to fix it as well, not just the cause. – James Funk Apr 19 '16 at 00:17

score 0 · Answer 3 · answered Jan 26 '17 at 22:12

Completing @Apperside answer.

I fixed the error by adding the following lines to my build.gradle file of my app module:

android{

    packagingOptions {
            exclude 'com/google/appengine/repackaged/org/apache/commons/codec/language/bm/*'
            exclude 'com/google/appengine/repackaged/org/codehaus/jackson/impl/VERSION.txt'
            exclude 'com/google/appengine/repackaged/org/apache/commons/codec/language/*'
        }

    (...)  
}

DuplicateFileException error while building Android APK

3 Answers3

Linked