1

I am running my spark job on 4 nodes cluster, after some time i keep getting error that "No space left on device".

i checked the nodes du -h -s * command and it shows stdout folder as full.

$$ du -h -s *
   4.0K logs
   4.0K stderr
   25G  stdout

As you can see stdout folder is full, How can i stop it. How can i tell my cluster to stop writing debug and info logs to stdout?

Amit Kumar
  • 825
  • 1
  • 9
  • 19
  • i think the question is duplicated, http://stackoverflow.com/questions/27781187/how-to-stop-messages-displaying-on-spark-console – Mehrez Apr 14 '17 at 10:41

0 Answers0