12

I tried copying files from my local disk to hdfs . At first it gave SafeModeException. While searching for solution I read that the problem does not appear if one executes same command again. So I tried again and it didn't gave exception.

hduser@saket:/usr/local/hadoop$ bin/hadoop dfs -copyFromLocal /tmp/gutenberg/ /user/hduser/gutenberg
copyFromLocal: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /user/hduser/gutenberg. Name node is in safe mode.
hduser@saket:/usr/local/hadoop$ bin/hadoop dfs -copyFromLocal /tmp/gutenberg/ /user/hduser/gutenberg

Why is this happening?. Should I keep safemode off by using this code?

hadoop dfs -safemode leave
eLRuLL
  • 18,488
  • 9
  • 73
  • 99
saket
  • 694
  • 5
  • 10
  • 20

2 Answers2

18

NameNode is in safemode until configured percent of blocks reported to be online by the data nodes. It can be configured by parameter dfs.namenode.safemode.threshold-pct in the hdfs-site.xml

For small / development clusters, where you have very few blocks - it makes sense to make this parameter lower then its default 0.9999f value. Otherwise 1 missing block can lead to system to hang in safemode.

AdrieanKhisbe
  • 3,899
  • 8
  • 37
  • 45
David Gruzman
  • 7,900
  • 1
  • 28
  • 30
15

Go to the hadoop path into bin(in my system /usr/local/hadoop/bin/),

cd /usr/local/hadoop/bin/

Check there is a file hadoop,

hadoopuser@arul-PC:/usr/local/hadoop/bin$ ls

the o/p will be,

hadoop            hadoop-daemons.sh  start-all.sh       start-jobhistoryserver.sh  stop-balancer.sh          stop-mapred.sh
hadoop-config.sh  rcc                start-balancer.sh  start-mapred.sh            stop-dfs.sh               task-controller
hadoop-daemon.sh  slaves.sh          start-dfs.sh       stop-all.sh                stop-jobhistoryserver.sh

Then you have to off safe mode by using command ./hadoop dfsadmin -safemode leave,

hadoopuser@arul-PC:/usr/local/hadoop/bin$ ./hadoop dfsadmin -safemode leave

you will get response as,

Safe mode is OFF

Note: I created Hadoop user with the name of hadoopuser.

A J
  • 492
  • 1
  • 7
  • 24