0
df
 .coalesce(1)
 .write
 .format("com.crealytics.spark.excel")
 .option("useHeader", "true")
 .option("header", "true")
 .mode(SaveMode.Append)
 .save(s"s3://$bucket/$etlFolderPrefix/a.xlsx")

ERROR [main] glue.ProcessLauncher (Logging.scala:logError(94)): Exception in User Class java.lang.NoSuchMethodError: org.apache.commons.io.IOUtils.byteArray(I)[B at org.apache.commons.io.output.AbstractByteArrayOutputStream.needNewBuffer(AbstractByteArrayOutputStream.java:104) at org.apache.commons.io.output.UnsynchronizedByteArrayOutputStream.(UnsynchronizedByteArrayOutputStream.java:51) at org.apache.commons.io.output.UnsynchronizedByteArrayOutputStream.(UnsynchronizedByteArrayOutputStream.java:38) at shadeio.poi.xssf.usermodel.XSSFWorkbook.newPackage(XSSFWorkbook.java:528) at shadeio.poi.xssf.usermodel.XSSFWorkbook.(XSSFWorkbook.java:245) at shadeio.poi.xssf.usermodel.XSSFWorkbook.(XSSFWorkbook.java:241) at shadeio.poi.xssf.usermodel.XSSFWorkbook.(XSSFWorkbook.java:229) at com.crealytics.spark.excel.ExcelFileSaver.save(ExcelFileSaver.scala:45) at com.crealytics.spark.excel.DefaultSource.createRelation Exception in User Class: java.lang.NoClassDefFoundError : org/apache/commons/io/output/UnsynchronizedByteArrayOutputStream

Tried with different versions, currently using spark version 2.12

Dmytro Mitin
  • 48,194
  • 3
  • 28
  • 66
sangeet singh
  • 21
  • 3
  • `NoClassDefFoundError` https://stackoverflow.com/questions/34413/why-am-i-getting-a-noclassdeffounderror-in-java https://stackoverflow.com/questions/1457863/what-causes-and-what-are-the-differences-between-noclassdeffounderror-and-classn – Dmytro Mitin Jan 04 '23 at 12:08
  • `NoSuchMethodError` https://stackoverflow.com/questions/35186/how-do-i-fix-a-nosuchmethoderror https://stackoverflow.com/questions/8168052/java-lang-nosuchmethoderror-when-the-method-definitely-exists https://stackoverflow.com/questions/59706633/nosuchmethoderror-java – Dmytro Mitin Jan 04 '23 at 12:09
  • Can be an issue with dependencies. Please provide details how you're running your code. [How to create a Minimal, Reproducible Example](https://stackoverflow.com/help/minimal-reproducible-example) – Dmytro Mitin Jan 04 '23 at 12:11
  • Can you elaborate more, we are using spark 2.12 and spark-excel 2.12. Looks like both are compatible with each other already? – sangeet singh Jan 04 '23 at 17:19
  • Please provide details how you're running your code. What is "using spark 2.12 and spark-excel 2.12"? Using how? Is it sbt or Maven project? What is build file (`build.sbt` or `pom.xml`)? Or are you running some command? What command? Please provide MCVE. – Dmytro Mitin Jan 04 '23 at 18:06
  • Running it locally, using maven i.e pom.xml – sangeet singh Jan 04 '23 at 21:56
  • Then please add `pom.xml` (especially dependencies) to the question. – Dmytro Mitin Jan 04 '23 at 22:05
  • *"we are using spark 2.12 and spark-excel 2.12"* I suspect those are not the only dependencies and there are their dependencies too. – Dmytro Mitin Jan 10 '23 at 16:17

0 Answers0