1

I am trying to run spark in standalone mode on CentOs, and trying to read text file from local file system using

scala > var text = sc.textFile("/home/Documents/test.txt")

but it is giving exception -

Caused by: java.net.ConnectException: Connection refused.

But when I am changing the code as :

 scala > var text = sc.textFile("file:///home/Documents/test.txt")

I am able to read it.
My question is :

when tried reading file without file:///, is it looking the file in HDFS file system ? and it yes then is HDFS is default FS where spark search for the file ?

Ramesh Maharjan
  • 41,071
  • 6
  • 69
  • 97
Rex
  • 149
  • 3
  • 10

0 Answers0