1

I tried to install pgAgent, but since it is not supported on Amazon I don't know how to schedule postgres jobs without going with Cron jobs and psql directly. Here is what I got on Amazon RDS: enter image description here

The following command gave the same result:

CREATE EXTENSION pg_cron;
David
  • 4,786
  • 11
  • 52
  • 80
  • Same problem discussed here: http://stackoverflow.com/questions/42564910/how-to-execute-scheduled-sql-script-on-redshift/42567999#42567999 – systemjack Mar 03 '17 at 18:29

3 Answers3

2

I have total of three options right now on top of my head for this: 1.)AWS Lambda 2.)AWS Glue 3.)Any small EC2 instance (Linux/Windows)

1.)AWS Lambda: you can use postgres connectvity python module like pg8000 or psycopg2, to connect and create cursor to your target RDS. and you can pass your sql jobs code /your SQL statements as an input to lambda. If they are very few, you can just code the whole job in your lambda, if not you can pass it to lambda as a input using DynamoDB. You can have a cron schedule using cloudwatch event, so that it will trigger lambda whenever you need. Required tools: DynamoDB, AWS Lambda, Python, Postgres python connectivity module.

2.)AWS Glue AWS Glue also works almost same. You have a option to connect to your RDS DB directly there and you can schedule your jobs there.

3.)Ec2 instance: Create any small size ec2 instance, either windows or linux and have setup your cron/bat jobs.

winnervc
  • 799
  • 7
  • 10
1

You could use Amazon CloudWatch Events to trigger a Lambda function on a schedule, but it can only run for a maximum of 15 minutes (https://aws.amazon.com/about-aws/whats-new/2018/10/aws-lambda-supports-functions-that-can-run-up-to-15-minutes/?nc1=h_ls).

You could also run a t2.nano Amazon EC2 instance (about $50/year On-Demand, or $34/year as a Reserved Instance) to run regular cron jobs.

dp6000
  • 473
  • 5
  • 15
John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
1

On October 10th, 2018, AWS Lambda launched support for long running functions. Customers can now configure their AWS Lambda functions to run up to 15 minutes per execution. Previously, the maximum execution time (timeout) for a Lambda function was 5 minutes. Using longer running functions, a highly requested feature, customers can perform big data analysis, bulk data transformation, batch event processing, and statistical computations more efficiently.