Actually whenever we are creating partition in hive, how much size is getting allocated. Is it same like 1 Block for 1 partition i.e, 128 MB ? And also is it possible to change the size of partition?
Asked
Active
Viewed 174 times
1 Answers
0
In Hive partition is a folder (and of course Hive metadata about partition and it's location), no size is allocated initially. The size of partition is the total size of files located in the partition location folder.
Read also this: https://stackoverflow.com/a/47720850/2700344
leftjoin
- 36,950
- 8
- 57
- 116
-
Thanks.. Just want to confirm if i am having 300MB for single partition key So it will allocate approximately 3 blocks(partition) ? – Karpagaraj Durairaj Mar 23 '19 at 11:53
-
@KarpagarajDurairaj Yes, and It also depends on the compression and file format. It can be compressed > 3 times – leftjoin Mar 23 '19 at 13:02