0

before I this work, I saved my files using pyspark. cause they were so big to handling with python. saved as Pickle but seems like dir, idk.

And after works did done, I call with sc.pickleFile(file_name) like this, and this error occurs.

Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: {filename}

Files( DIRs? ) in one Directory, and if i entered specific one, the file names like part-00000, part-00001 ...

I've changed pythonpath, or work env ( opened jupyter in Dir ). But All doesn't work i can't handling my files :(

enter image description here

MattDMo
  • 100,794
  • 21
  • 241
  • 231
Eunbin Park
  • 1
  • Seems pretty straightforward. `java.net.URISyntaxException: Relative path in absolute URI: {filename}`. So, when you call `sc.pickleFile(file_name)`, make sure `file_name` is an absolute path. – MattDMo Sep 05 '21 at 02:14
  • I already put absolute path to file_name. But same error comes :( – Eunbin Park Sep 06 '21 at 03:04
  • i solve this... omg i'm so stupid. https://stackoverflow.com/questions/25334604/hadoop-java-net-urisyntaxexception-relative-path-in-absolute-uri-rsrchbase-co/25336223 – Eunbin Park Sep 07 '21 at 01:08

0 Answers0