1

I am looking for some way to directly export the SQL query results to a CSV file from AWS lambda. I have found this similar question - Exporting table from Amazon RDS into a csv file. But it will not work with the AWS Golang API.

Actually, I want to schedule a lambda function which will daily query some of the views/tables from RDS(SQL Server) and put it to the S3 bucket in CSV format. So, I want to directly download the query results in the CSV form in the lambda and then upload it to S3.

I have also found data pipeline service of AWS to copy RDS data to S3 directly, but I am not sure if I can make use of it here.

It would be helpful if anyone can suggest me the right process to do it and references to implement it.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Rahul Satal
  • 2,107
  • 3
  • 32
  • 53
  • 1
    Are you lloking for this `query results to a CSV file in the local machine` or `I want to schedule a lambda function` (I assume you refer to AWS lambda). Running code in cloud and in local machine are two different things. – Dmitry Harnitski Jan 05 '20 at 15:34
  • @DmitryHarnitski Sorry for the confusion. I want to schedule a lambda function. – Rahul Satal Jan 05 '20 at 15:59

1 Answers1

1

You can transfer files between a DB instance running Amazon RDS for SQL Server and an Amazon S3 bucket. By doing this, you can use Amazon S3 with SQL Server features such as BULK INSERT. For example, you can download .csv, .xml, .txt, and other files from Amazon S3 to the DB instance host and import the data from D:\S3\into the database. All files are stored in D:\S3\ on the DB instance

Reeference:

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/User.SQLServer.Options.S3-integration.html

AWS PS
  • 4,420
  • 1
  • 9
  • 22
  • Thanks for the response. But I want to query data from RDS(SQL Server) and not Aurora. – Rahul Satal Jan 05 '20 at 16:01
  • Following the documentation, I have created the IAM role and set up it into the RDS instance. But even after spending so much time, I couldn't create a stored procedure to export the result in CSV. I am getting this error - https://stackoverflow.com/questions/59602890/unable-to-create-a-stored-procedure-on-ssms – Rahul Satal Jan 05 '20 at 18:23