0

I have a zip file in FTP server. I want to retrieve the zip file and unzip it list the files in this zip, select some files and transfer it again to another folder in the FTP server. How we can do this with pyspark or spark Scala?

I tried this with Scala :

val df = spark.read.
            format("com.springml.spark.sftp").
            option("host", "SFTP_HOST").
            option("username", "SFTP_USER").
            option("password", "****").
            option("fileType", "csv").
            option("delimiter", ";").
            option("quote", "\"").
            option("escape", "\\").
            option("multiLine", "true").
            option("inferSchema", "true")

but it does not work for me.

jwvh
  • 50,871
  • 7
  • 38
  • 64
Omayma HARBAOUI
  • 47
  • 5
  • 1
    Does this answer your question? [How to open/stream .zip files through Spark?](https://stackoverflow.com/questions/28569788/how-to-open-stream-zip-files-through-spark) – Jasper-M Aug 27 '21 at 08:57
  • yes but my problem is that how we can connect to FTP server and navigate into to find the zip file with pyspark – Omayma HARBAOUI Aug 27 '21 at 09:01

0 Answers0