0

I know one can use the --jars option with spark-submit, but in that case, I have to give full paths for all the files. Isn't there a simpler way where I can just put all my jar files in a folder and submit the whole folder?

MetallicPriest
  • 29,191
  • 52
  • 200
  • 356
  • 1
    you could use `--conf spark.driver.extraClassPath=/* --conf spark.executor.extraClassPath=/*` with spark-submit – shanmuga Mar 06 '19 at 10:52
  • Works fine on Linux, but I am having problem on Windows. I didn't try your suggestion but add paths in spark.default.conf. But I guess I would have the same problem with spark-submit as well. – MetallicPriest Mar 06 '19 at 10:56

0 Answers0