5

In Scala, as an experiment I create a sequence file on Tachyon using Spark and read it back in. I want to delete the file from Tachyon using the Spark script also.

val rdd =  sc.parallelize(Array(("a",2), ("b",3), ("c",1)))
rdd.saveAsSequenceFile("tachyon://127.0.0.1:19998/files/123.sf2")
val rdd2 = sc.sequenceFile[String,Int]("tachyon://127.0.0.1:19998/files/123.sf2")

I don't understand the Scala language very well and I cannot find a reference about file path manipulation. I did find a way of somehow using Java in Scala to do this, but I cannot get it to work using Tachyon.

import java.io._
new File("tachyon://127.0.0.1:19998/files/123.sf2").delete()
dtolnay
  • 9,621
  • 5
  • 41
  • 62
bjjer
  • 983
  • 1
  • 7
  • 7
  • I've had a similar problem, where I tried to clean up a file path after using it by deleting all the files I created. But for some reason it refused to delete all the files. Turns out it refused to delete the file if it was being used by a different process or program. That might be your problem here – Electric Coffee Jul 19 '14 at 08:22
  • You need to use the Tachyon API. – BAR Oct 25 '15 at 08:59

1 Answers1

-1

There are different approaches, e.g.:

  • CLI:

    ./bin/tachyon tfs rm filePath

More info: http://tachyon-project.org/Command-Line-Interface.html

  • API: 

    TachyonFS sTachyonClient = TachyonFS.get(args[0]);
sTachyonClient.delete(filePath, true);

More info: https://github.com/amplab/tachyon/blob/master/core/src/main/java/tachyon/examples/BasicOperations.java

Prasad Khode
  • 6,602
  • 11
  • 44
  • 59
sltz
  • 34
  • 3