Gradle exclude java class from lib replaced by own class to avoid duplicate

Question

In Android Studio, there is a specific file (src/org/luaj/vm2/lib/jse/JavaMethod.java) that I need to overwrite from a package that is pulled in via Gradle (dependencies {compile 'org.luaj:luaj-jse:3.0.1'}).

I copied the file into my source directory with the exact same path and made my changes to it. This was working fine for an individual JUnit test case that was using it. It also looks like it is working for a normal compile of my project (unable to easily confirm at the moment).

However, when I try to run all my tests at once via a configuration of ProjectType="Android Tests", I get Error:Error converting bytecode to dex: Cause: com.android.dex.DexException: Multiple dex files define Lorg/luaj/vm2/lib/jse/JavaMethod$Overload;.

Is there a specific task or command that I need to add to my Gradle file to make sure the project selects the file in my local source directory? I tried the Copy task and the sourceSets->main->java->exclude command, but neither seemed to work (I may have done them wrong). I also tried the "exclude module/group" directive under "compile" from this post.

The non-default settings for the Run/Debug Confirmation:

• Type=Android Tests
• Module=My module
• Test: All in package
• Package: "test"

All my JUnit test cases are in the "test" package.

Any answer that gets this to work is fine. If not Gradle, perhaps something in the android manifest or the local source file itself.

[Edit on 2016-07-24] The error is also happening on a normal compile when my android emulator is running lower APIs. API 16 and 19 error out, but API 23 does not.

Answer 1

issue: when linking your app the linker finds two versions

• org.luaj:luaj-jse:3.0.1:org.luaj.vm2.lib.jse.JavaMethod and
• {localProject}:org.luaj.vm2.lib.jse.JavaMethod

howto fix: tell gradle to exclude org.luaj:luaj-jse:3.0.1:org.luaj.vm2.lib.jse.JavaMethod from building

android {
    packagingOptions {
        exclude '**/JavaMethod.class'
    }
}

I have not tried this with "exclude class" but it works for removing duplicate gpl license files a la "COPYING".

If this "exclude" does not work you can

• download the lib org.luaj:luaj-jse:3.0.1 to the local libs folder,
• open jar/aar with a zip-app and manually remove the duplicate class.
• remove org.luaj:luaj-jse:3.0.1 from dependencies since this is now loaded from lib folder

Answer 2

I am not completely sure I understand your problem; however, it sounds like a classpath ordering issue, not really a file overwrite one.

AFAIK, gradle does not make a 'guarantee' on the ordering from a 'dependencies' section, save for that it will be repeatable. As you are compiling a version of file that you want to customize, to make your test/system use that file, it must come earlier in the classpath than the jar file it is duplicated from.

Fortunately, gradle does allow a fairly easy method of 'prepending' to the classpath:

sourceSets.main.compileClasspath = file("path/to/builddir/named/classes") + sourceSets.main.compileClasspath

I don't know enough about your system to define that better. However, you should be able to easily customize to your needs. That is, you can change the 'compile' to one of the other classpath (runtime, testRuntime, etc) if needed. Also, you can specify the jarfile you build rather than the classes directory if that is better solution. Just remember, it may not be optimal, but it is fairly harmless to have something specified twice in the classpath definition.

Answer 3

This is rather convoluted but it is technically feasible. However it's not a single task as asked by the poster:

1. Exclude said dependency from build.gradle and make sure it's not indirectly included by another jar (hint: use ./gradlew dependencies to check it)
2. create a gradle task that downloads said dependency in a known folder
3. unpack such jar, remove offending .class file
4. include folder as compile dependency

If it's safe to assume that you're using Linux/Mac you can run a simple command line on item 3, it's only using widely available commands:

mkdir newFolder ; cd newFolder ; jar xf $filename ; rm $offendingFilePath

If you don't care about automatic dependency management you can download the jar file with curl, which I believe to be widely available on both linux and mac.

curl http://somehost.com/some.jar -o some.jar

For a more robust implementation you can substitute such simple command lines with groovy/java code. It's interesting to know that gradle can be seen as a superset of groovy, which is arguable a superset of java in most ways. That means you can put java/groovy code pretty much anywhere into a gradle.build file. It's not clean but it's effective, and it's just another option.

For 4 you can have something along either

sourceSets.main.java.srcDirs += ["newFolder/class"]

at the root level of build.gradle, or

dependencies {
. . . 
   compile fileTree(dir: 'newFolder', include: ['*.class'])
. . . 

Answer 4

This is what I ended up adding after Fabio's suggestion:

//Get LUAJ
buildscript { dependencies { classpath 'de.undercouch:gradle-download-task:3.1.1' }}
apply plugin: 'de.undercouch.download'
task GetLuaJ {
    //Configure
    def JARDownloadURL='http://central.maven.org/maven2/org/luaj/luaj-jse/3.0.1/luaj-jse-3.0.1.jar' //compile 'org.luaj:luaj-jse:3.0.1'
    def BaseDir="$projectDir/luaj"
    def ExtractToDir='class'
    def ConfirmAlreadyDownloadedFile="$BaseDir/$ExtractToDir/lua.class"
    def JarFileName=JARDownloadURL.substring(JARDownloadURL.lastIndexOf('/')+1)
    def ClassesToDeleteDir="$BaseDir/$ExtractToDir/org/luaj/vm2/lib/jse"
    def ClassNamesToDelete=["JavaMethod", "LuajavaLib"]

    //Only run if LuaJ does not already exist
    if (!file(ConfirmAlreadyDownloadedFile).exists()) {
        //Download and extract the source files to /luaj
        println 'Setting up LuaJ' //TODO: For some reason, print statements are not working when the "copy" directive is included below
        mkdir BaseDir
        download {
            src JARDownloadURL
            dest BaseDir
        }
        copy {
            from(zipTree("$BaseDir/$JarFileName"))
            into("$BaseDir/$ExtractToDir")
        }

        //Remove the unneeded class files
        ClassNamesToDelete=ClassNamesToDelete.join("|")
        file(ClassesToDeleteDir).listFiles().each {
            if(it.getPath().replace('\\', '/').matches('^.*?/(?:'+ClassNamesToDelete+')[^/]*\\.class$')) {
                println "Deleting: $it"
                it.delete()
            }
        }
    }
}

I'll upload a version that works directly with the jar later.

Answer 5

Another solution if we got then source jar:

task downloadAndCopy {
    def downloadDir = "${buildDir}/downloads"
    def generatedSrcDir = "${buildDir}/depSrc"
    copy {
        from(configurations.detachedConfiguration(dependencies.add('implementation', 'xxx:source')))
        file(downloadDir).mkdirs()
        into(downloadDir)
    }

    println("downloading file into ${downloadDir}")

    fileTree(downloadDir).visit { FileVisitDetails details ->
        if (!details.file.name.endsWith("jar")) {
            println("ignore ${details.file.name}")
            return
        }
        println("downloaded ${details.file.name}")
        def srcFiles = zipTree(details.file).matching {
            include "**/*.java"
            exclude "**/NeedEclude*java"
        }
        srcFiles.visit {FileVisitDetails sourceFile ->
            println("include ${sourceFile}")
        }

        copy {
            from(srcFiles)
            into(generatedSrcDir)
        }
    }
}

and remember to add depSrc to srcDirs

android {
  sourceSets {
    `main.java.srcDirs = ['src/main/java', "${buildDir}/depSrc"] 
  }
}