sbt assembly fails due to different file contents found

Question

I am trying to build a project in GitLab. In gitlab-ci.yml, I ran sbt assembly and encountered annoying exception.

[error] (soda/*:assembly) deduplicate: different file contents found in the following:
[error] /root/.ivy2/cache/io.netty/netty-buffer/jars/netty-buffer-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-common/jars/netty-common-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-codec-http/jars/netty-codec-http-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-codec/jars/netty-codec-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-transport/jars/netty-transport-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-handler/jars/netty-handler-4.0.42.Final.jar:META-INF/io.netty.versions.properties
[error] /root/.ivy2/cache/io.netty/netty-transport-native-epoll/jars/netty-transport-native-epoll-4.0.42.Final-linux-x86_64.jar:META-INF/io.netty.versions.properties

I tried to follow the instruction in sbt-assembly: deduplication found error, and it seems like the MergeStrategy is in place, however the exception remain:

[info] Merging files...
[warn] Merging 'NOTICE' with strategy 'rename'
[warn] Merging 'README' with strategy 'rename'
[warn] Merging 'META-INF/NOTICE.txt' with strategy 'rename'
[warn] Merging 'license/NOTICE' with strategy 'rename'
[warn] Merging 'META-INF/NOTICE' with strategy 'rename'
[warn] Merging 'org/xerial/snappy/native/README' with strategy 'rename'
[warn] Merging 'license' with strategy 'rename'
[warn] Merging 'license/LICENSE' with strategy 'rename'
[warn] Merging 'META-INF/license' with strategy 'rename'
[warn] Merging 'META-INF/LICENSE.txt' with strategy 'rename'
[warn] Merging 'LICENSE.txt' with strategy 'rename'
[warn] Merging 'META-INF/LICENSE' with strategy 'rename'
[warn] Merging 'LICENSE' with strategy 'rename'
[warn] Merging 'META-INF/DEPENDENCIES' with strategy 'discard'
[warn] Merging 'META-INF/INDEX.LIST' with strategy 'discard'
[warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'

I also tried to change the sbt version, but the problem stays.

Any help with how can I resolve this issue will be great.

score 0 · Answer 1 · answered Apr 23 '19 at 07:54

0

Looked everywhere could not find any resolution. Check the following resolution if that works for you, It did not work for me: https://github.com/sbt/sbt-assembly/issues/297

answered Apr 23 '19 at 07:54

Saurabh Kolte

1
1

If this is not a solution why post it? – Tamir Klein Apr 23 '19 at 07:56
I tried it as well, but it keeps failing with the same error. Anyway, thanks for replying. – EyalP Apr 23 '19 at 08:33

score 0 · Answer 2 · answered Apr 23 '19 at 08:27

I once had a similarproblem with some Java packages and log4j, What I did was to create a custom merge strategy in which I only choosed the first occurrence. Maybe you can adapt that code to work for you:

assemblyMergeStrategy in assembly := {
//  case PathList("javax", "servlet", xs @ _*)         => MergeStrategy.first
  case PathList("org", "apache", "commons", xs @ _*)        =>
 //   println(s"$xs")
    MergeStrategy.first
  case PathList(ps @ _*) if ps.last endsWith ".html" => MergeStrategy.first
  case "application.conf"                            => MergeStrategy.concat
  case "log4j.properties"                            => MergeStrategy.first
  case "unwanted.txt"                                => MergeStrategy.discard
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

Maybe if you change

 case PathList("org", "apache", "commons", xs @ _*)        =>
 //   println(s"$xs")
    MergeStrategy.first

to your problematic packages (netty) you manage to solve it:

 case PathList("io", "netty", xs @ _*)        =>
 //   println(s"$xs")
    MergeStrategy.first

Thanks for your reply, but it didn't help. The link I attached suggest something pretty similar: assemblyMergeStrategy in assembly := { case PathList("META-INF", xs @ _*) => MergeStrategy.discard case x => MergeStrategy.first } anyhow, I tried your suggestion too. — EyalP, Apr 23 '19 at 08:56
It is a little tricky to solve this types of problems, at least in my experience. — Alejandro Alcalde, Apr 23 '19 at 09:28

EyalP · Accepted Answer · 2019-06-20T10:04:02.957

Eventually I found a workaround for this problem. Since its a play project, I could use activator to wrap it up and create a package. I used the following code in order to create the package: wget https://downloads.typesafe.com/typesafe-activator/1.3.7/typesafe-activator-1.3.7.zip && unzip typesafe-activator-1.3.7.zip 1> /dev/null activator-dist-1.3.7/activator -Dsbt.log.noformat=true -java-home /usr/lib/jvm/java-1.8-openjdk package clean compile test dist

sbt assembly fails due to different file contents found

3 Answers3

Linked