Viewing the number of blocks for a file in hadoop

Question

How can I view how many blocks has a file been broken into, in a Hadoop file system?

score 45 · Answer 1 · edited Jul 23 '15 at 02:11

45

We can use hadoop file system check command to know the blocks for the specific file.

Below is the command:

hadoop fsck [path] [options]

To view the blocks for the specific file :

hadoop fsck /path/to/file -files -blocks

edited Jul 23 '15 at 02:11

Phani

answered Jun 23 '12 at 11:03

score 4 · Answer 2 · answered Sep 16 '15 at 20:57

4

hadoop fsck filetopath

used the above commad in CDH 5. Got the below Error.

hadoop-hdfs/bin/hdfs: line 262: exec: : not found

Use the below command and it worked good

hdfs fsck filetopath

answered Sep 16 '15 at 20:57

yoga

score 3 · Answer 3 · answered Nov 04 '19 at 15:35

3

It is always a good idea to use hdfs instead of hadoop as 'hadoop' version is deprecated.

Here is the command with hdfs and to find the details on a file named 'test.txt' in the root, you would write

hdfs fsck /test.txt -files -blocks -locations

answered Nov 04 '19 at 15:35

user1795667

score -4 · Answer 4 · answered May 29 '13 at 11:54

-4

This should work..

hadoop fs -stat "%o" /path/to/file

answered May 29 '13 at 11:54

user1514507

`%o` is the block size, not the number of blocks, per http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/FileSystemShell.html#stat – Nickolay May 25 '15 at 02:34

4 Answers4