I have a question about the name node High Availability. Name node is so important because it stores all the metadata, if it is down, the whole Hadoop Cluster will be down as well. So is there any good way to approach the name node High Availability, for example there is backup name node that can take over when the primary name node fails? (now I use Hadoop 1.1.2)
Asked
Active
Viewed 2,578 times
0
-
I don't know the specific downtime...in my opinion, the sooner the better – user2597504 Aug 12 '13 at 19:17
-
Consider looking at this article: http://hadoopblog.blogspot.in/2010/02/hadoop-namenode-high-availability.html – SSaikia_JtheRocker Aug 12 '13 at 19:20
-
Have a look at : http://stackoverflow.com/questions/4502275/hadoop-namenode-single-point-of-failure/34850278#34850278 – Ravindra babu Jan 18 '16 at 08:47
1 Answers
0
For ASF Hadoop 1.1.2, there are no solid NameNode HA options. These were released for 2.0 and are included in popular distributions like Cloudera's CDH4.
The options for NameNode HA include running a primary NameNode and a hot standby NameNode. They share an edits log, either on a NFS mount, or through quorum journal mode in HDFS itself. The former gives you the benefit of having an external source for storing your HDFS metadata, while the latter gives you the benefit of having no dependencies external to Hadoop.
Personally, I like the NFS option, as you can easily snapshot/backup the data resident the file server. The disadvantage to this approach is potentially inconsistent performance in terms of latency.
For more detail, check out the following articles:
jtravaglini
- 1,676
- 11
- 19
-
If we just restart the name node when it is down, can it recovery everything? If we can do like this, why do we need to set up a standby name node? – user2597504 Aug 13 '13 at 16:42
-
The standby (HA) NameNode can be configured to automatically start in the event the primary NameNode fails. Further, if the fsimage on the primary NameNode becomes corrupted, there is a warm copy ready to go on the HA NameNode. – jtravaglini Aug 13 '13 at 16:44
-
Ok. But my question is that if we just restart the name node, can it recovery everything, right? – user2597504 Aug 13 '13 at 16:56
-
No, not necessarily. Data could be missing/corrupt. Not all NameNode errors are transient - in fact many of them or not. I would suggest researching the Hadoop Administrator's guide for more info. – jtravaglini Aug 13 '13 at 16:57