Sample Data:
Table has partitioned with year and month and day I want to get the files which are created in last 100 days and then I need to delete those files.
Asked
Active
Viewed 90 times
0
-
If you actually want to remove partitions older than 100 days, then see this answer: https://stackoverflow.com/a/56646879/2700344 All you need is to modify slightly calculation of min year, month and day to be 100 days ago instead of 7 days. And if you really want to remove files, keeping partitioning as is, then you need to list all files in each partition, extracting each file create date (it can be different from partition date) and remove old files. – leftjoin Jan 05 '20 at 08:46
-
Does this answer your question? [HIVE select count(\*) non null returns higher value than select count(\*)](https://stackoverflow.com/questions/39912784/hive-select-count-non-null-returns-higher-value-than-select-count) – damientseng Jan 06 '20 at 02:11
-
@gaccio the link you provided as a duplicate is not relevant, it seems mistake – leftjoin Jan 07 '20 at 08:06
-
@leftjoin You're right, seems like a mistake, I meant sth else :) That comment might have been auto-generated as I reported that this question may be a duplicate to this: https://stackoverflow.com/questions/59592612/i-need-to-select-and-delete-the-hdfs-files-older-than-100-days-in-hive-partition – damientseng Jan 07 '20 at 08:16