0

From all Redshift documentation / forums I've read, I understand the files in path are deleted before the unload.

From UNLOAD - Amazon Redshift:

CLEANPATH
The CLEANPATH option removes existing files located in the Amazon S3 path specified in the TO clause before unloading files to the specified location.

However, when I am testing this, the original files are still there and don't get deleted until after the unload is complete, except for those that are 'overwritten', is this how it works, or am I doing something wrong? It seems to take several seconds after the unload completes, and I'd be afraid that with my next step, ie, copy, some of those files which might not yet have deleted, will cause dupes in my target.

Thanks for your answer !!!

Summary: I ran unload with maxfile 250, then ran again with maxfile 500, and some of the previous files showed on the S3 bucket for several seconds after the completion of the unload.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Mme Butterfly
  • 1
  • It might be helpful if you link to the documentation you are talking about. I have never heard of "S3 CLEANPATH" and a search just turns up something related to Amazon Redshift. Are you using Redshift? You haven't mentioned it or tagged it in your question at all. – Mark B Apr 07 '23 at 16:23

0 Answers0