2

I'm learning Hadoop and I've installed singe node Hadoop cluster (version: 2.7.3) properly on my Mac. I already uploaded few text files in my HDFS and today when I'm trying to create a new directory in HDFS, it's showing

mkdir: Cannot create directory /horton. Name node is in safe mode.

Then I searched "What is safe mode in Hadoop?". I came to know that Namenode enters into safemode when there is shortage of memory. As a result the HDFS becomes readable only. That means one can not create any additional directory or file in the HDFS. To come out of the safemode, the following command is used:

hadoop dfsadmin -safemode leave

I've almost 150 GB free disk space and in my HDFS I've created 5 folders with some text files uploaded with it. So, I want to know how my HDFS storage gets full in just few folders and text files and if there are some other reasons for going NameNode in safe mode, please let me know the reasons.

Thank you!

arctic_Oak
  • 974
  • 2
  • 13
  • 29
  • 1
    You should read the namenode logs on disk to see if you can find at what time safemode was enabled. Also, if you've shutdown your computer while HDFS is running, it's likely to get corrupted, which is another reason for safemode – OneCricketeer Feb 17 '18 at 14:38
  • 1
    Also, default replication factor is 3, and all data is stored under /tmp. Both of those should be changed 1) so your data isn't wiped 2) because you don't need replicas on a single machine/disk – OneCricketeer Feb 17 '18 at 14:43
  • Thanks @cricket_007 can you tell me one thing that in single node cluster can we start more than one data node other than using virtual machines. – arctic_Oak Feb 17 '18 at 19:09
  • 1
    The datanode requires a number of open ports, and you cannot open those ports more than once on one network adapter. – OneCricketeer Feb 17 '18 at 22:08
  • I found one way of doing that but I need your help actually in line `bin/hadoop-daemon.sh --script bin/hdfs $1 datanode $DN_CONF_OPTS`( refer to the below link). The path bin/hadoop-daemon.sh is throwing error and I don't know the exact substitute and even I'm understanding anything at this point https://bigdata.wordpress.com/2010/05/27/hadoop-cookbook-4-how-to-run-multiple-data-nodes-on-one-machine/ @cricket_007 – arctic_Oak Apr 22 '18 at 20:17
  • I don't know the substitutes, either, sorry. You can open the `bin/hadoop-daemon.sh` and add a `set -xv` after the shebang line to debug it though – OneCricketeer Apr 22 '18 at 20:40

0 Answers0