3

I have a function that takes a query and saves it to an s3 bucket, but it comes as a long and non-meaningful name. Is there a way to set the output file name when saving to s3 via running an Athena query with Boto3? I would like to edit this function to save the output to a specific name on s3.

Here is my code to run an Athena query: 

def run_query(query, database, s3_output):    
response = client.start_query_execution(
    QueryString=query,
    QueryExecutionContext={
        'Database': database
        },
    ResultConfiguration={
        'OutputLocation': s3_output,
        }
    )
print('Execution ID: ' + response['QueryExecutionId'])
return response
Joe S
  • 410
  • 6
  • 16
  • You could always copy the object to a new name & location after execution. – John Rotenstein Jan 14 '19 at 21:24
  • 2
    Possible duplicate of [How to change the name of the Athena results stored in S3?](https://stackoverflow.com/questions/51412005/how-to-change-the-name-of-the-athena-results-stored-in-s3) – jbgorski Jan 15 '19 at 07:27
  • The output of athena query is the following and you cannot change it. https://docs.aws.amazon.com/athena/latest/ug/querying.html But you can write the script to rename these output files. https://stackoverflow.com/questions/21184720/how-to-rename-files-and-folder-in-amazon-s3 – jbgorski Jan 15 '19 at 07:30
  • @JohnRotenstein that costs money, which is why I wanted to just save it as the right name to begin with... – Joe S Jan 15 '19 at 13:35
  • @j.b.gorski I disagree as I want to write a function in python that saves it to the correct name, not changes it after the fact. – Joe S Jan 15 '19 at 13:37
  • @Joe S As I wrote,it isn't possible, therefore you have to rename output files after a query execution. It is strange, but that's how it works. You can only change the path where the output files are generated. – jbgorski Jan 15 '19 at 13:43
  • @JoeS So, you are concerned about the cost of copying the object? The API call costs $0.005 per 1,000 requests. If you are concerned about storage cost, you can delete the original object (produced by Athena) after the copy, so there is no additional storage cost. Storage in S3 is $0.023 per GB. So, you're talking a few cents. – John Rotenstein Jan 15 '19 at 20:32

0 Answers0