0

I have a large table and in which I would like overwrite certain top level partitions. for e.g. I have table which is partitioned based on year and month, and I would like to overwrite partitions say from year 2000 to 2018.

How I can do that.

Note : I would not like to delete the previous table and overwrite entire table with new data.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Neeraj Bhadani
  • 2,930
  • 16
  • 26
  • What currently happens when you write your dataframe with overwrite mode? – OneCricketeer Jul 18 '18 at 12:45
  • I need to update certain records. For that I would like to delete the partition and would like to overwrite specific partition, so that I need not to overwrite the entire table. For e.g. I only would like to update the data for year 2000 - 2018, so I would like to delete the partition and insert the new one. let me know if their is any other way to do it. – Neeraj Bhadani Jul 18 '18 at 12:48
  • Again, what code do you currently have that isn't doing this? – OneCricketeer Jul 18 '18 at 12:50
  • I tried to delete multiple partitions using command "ALTER TABLE table_name DROP IF EXISTS PARTITION (partition_col > val)" but its not working. however it is working with "=". if that works with ">" probably i can try to delete them and insert new one. – Neeraj Bhadani Jul 18 '18 at 12:54

0 Answers0