1

I want to update partitions using below code.

msck repair table_name

(I can't use the other options such as refresh statement like that)

But I don't know the best way to update partitions.

1) I run that code every minutes.

2) I select the partitions using show command then if partitions doesn't exists then run that code.

show partitions table_name

Which is the best way to update partition (the other option is okay), so there is no limitation to search data ?

Would you give me an advice?

Bethlee
  • 825
  • 3
  • 17
  • 28
  • Possible duplicate:https://stackoverflow.com/questions/13815179/how-to-update-drop-a-hive-partition – Keshav Pradeep Ramanath May 24 '17 at 04:56
  • Thank you for your comment. But I want to know how to use msck repair command properly. The result of answer is same so it looks like similar but a little bit different. – Bethlee May 24 '17 at 05:11

2 Answers2

4

Command msck repair table_name is expensive one. You can use command ADD PARTITION.

E.g.

ALTER TABLE tblName ADD PARTITION (dt='2008-08-08', country='us') location '/path/to/us/part080808'

If you don't want to check if partitions exists or not, simply use IF NOT EXISTS. It will create partition if not exists.

ALTER TABLE tblName ADD IF NOT EXISTS PARTITION (dt='2008-08-08', country='us') location '/path/to/us/part080808'
Manish Saraf Bhardwaj
  • 1,038
  • 8
  • 27
  • Thank you! I use it than msck repair command. I have a question. What do you mean of "msck repair table_name expensive on"? I'm not familiar with hive. So would you give me some more explanation? – Bethlee May 24 '17 at 05:29
  • 1
    @Bethlee `msck` standsd for `metastore check`. So hive tries to validate the data on HDFS with the data in the metastore. If it sees any discrepancies in ,say, the partitions, it will try to add those partition all at once or one by one. – philantrovert May 24 '17 at 06:35
  • @philantrovert Thank you so much for explaining it! – Bethlee May 24 '17 at 07:12
0

another option would also be 

ALTER TABLE tblName UPDATE PARTITIONS;
Gerhard Brueckl
  • 708
  • 1
  • 9
  • 24