0

We have Hadoop cluster version HDP – 26 , on redhat machines version 7.x

We run the following command to capture the files that have corrupted blocks

Example 1

[root@master_3 ~]# su hdfs
[hdfs@master_3 root]$ hdfs fsck -list-corruptfileblocks 

blk_1097240344/localhdp/Rtrone/intercept_by_type/2018/4/10/16/2018_4_03_11_45.parquet/part-00002-be0f80a9-2c7c-4c50-b18d-db0f94a98cff.snly.parquet
blk_1097240348/localhdp/Rtrone/intkjd_country/2018/4/10/16/2018_4_03_11_45.parquet/part-00003-8600d0e2-c6b6-49b7-89cd-ef243cda4c5e.snly.parquet
The filesystem under path '/' has 2 CORRUPT files

so seems the files are:

/localhdp/Rtrone/intercept_by_type/2018/4/10/16/2018_4_03_11_45.parquet/part-00002-be0f80a9-2c7c-4c50-b18d-db0f94a98cff.snly.parquet
/localhdp/Rtrone/intkjd_country/2018/4/10/16/2018_4_03_11_45.parquet/part-00003-8600d0e2-c6b6-49b7-89cd-ef243cda4c5e.snly.parquet

searching on google , I understand that the procedure to handle the corrupted block in files should be as the following:

  1. This will delete the corrupted HDFS blocks:

    hdfs fsck / -delete

  2. run again the hdfs fsck -list-corruptfileblocks to find out if corrupted block deleted successfully

    hdfs fsck -list-corruptfileblocks

  3. if corrupted blocks still exists ( as example 1 ) then we need to delete the files as the following

    hdfs fs -rm /localhdp/Rtrone/intercept_by_type/2018/4/10/16/2018_4_03_11_45.parquet/part-00002-be0f80a9-2c7c-4c50-b18d-db0f94a98cff.snly.parquet

    hdfs fs -rm /localhdp/Rtrone/intkjd_country/2018/4/10/16/2018_4_03_11_45.parquet/part-00003-8600d0e2-c6b6-49b7-89cd-ef243cda4c5e.snly.parquet

am I right or I miss something?

Please let me know what need to add or update regarding my procedure

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
enodmilvado
  • 443
  • 1
  • 9
  • 20
  • Stack Overflow is a site for programming and development questions. This question appears to be off-topic because it is not about programming or development. See [What topics can I ask about here](http://stackoverflow.com/help/on-topic) in the Help Center. Perhaps [Super User](http://superuser.com/) or [Unix & Linux Stack Exchange](http://unix.stackexchange.com/) would be a better place to ask. – jww May 02 '18 at 01:08
  • I see in stack overflow many similar questions with the same subject as my question as - https://stackoverflow.com/questions/19205057/how-to-fix-corrupt-hdfs-files so what is the diff here why my question isn't fit for stackoverflow? – enodmilvado May 02 '18 at 03:36
  • The rules of the sites can change over the years. But given the question is server administration, not programming, it makes sense to be asked elsewhere. (even though Stackoverflow will almost always be top Google hit) – OneCricketeer May 02 '18 at 09:46
  • @cricket_007 since I not get answer until now , can you please help me to understand if my solution is the right approach? – enodmilvado May 02 '18 at 09:49
  • You need to delete the corrupted blocks, yes. If you list them, then you find them and delete. What's the issue? – OneCricketeer May 02 '18 at 09:52
  • @cricket_007 dose my steps 1-3 are ok? – enodmilvado May 02 '18 at 10:00
  • You're misusing the word "Does". And the correct way to say that is "Are my steps ok". But, as posted in the answer, and the link you have, yes... Delete the listed files – OneCricketeer May 02 '18 at 10:08

1 Answers1

0

The procedure is very simple.Follow

  1. Find out the corrupted blocks

      hdfs fsck / | egrep -v '^\.+$' | grep -v eplica

  2. Find out files related to corrupted blocks

     hdfs fsck /path/to/corrupt/file -locations -blocks -files

  3. Remove the corrupted File

     hdfs fs -rm /Corrupted_File_Path

After these steps rerun "hdfs fsck /" to see that missing blocks are gone.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Rockshell
  • 87
  • 1
  • 1
  • 8