Automatically retrieve JSON data via URL every X second and store in Amazon DynamoDB

Question

I'm looking for an easy way to retrieve and store JSON in Amazon DynamoDB. I'm getting the data via an URL and I would like to query the URL every X second - example: wget http://open-stocks.com/api/get-data-10:21:33.json

The time in the URL should match time of request - so that's dynamic.

I guess I could spin up an entire Linux server on AWS and write a Python script generating the URL, getting the data and push it to Amazon DynamoDB - but I would love a sort of existing service, which made me not worrying about server OS, cronjobs etc...

Any help to such service, perhaps directly via AWS?

I think scheduled AWS Lambdas could be a way forward - http://stackoverflow.com/questions/11616205/run-scheduled-task-in-aws-without-cron — Mirec Miskuf, Nov 12 '15 at 14:39

score 5 · Accepted Answer · edited May 23 '17 at 11:58

5

AWS Lambda now supports Scheduled Tasks. Since Lambda can make HTTP requests and write to DynamoDB, using Lambda should work and you don't have to worry about setting up an EC2 instance with a cron job just for that.

edited May 23 '17 at 11:58

Community

1
1

answered Nov 12 '15 at 14:02

Lucas Azevedo

1,097
15
36

1

Thanks - just what I was looking for :) – Kasper Nielsen Nov 12 '15 at 18:22

Automatically retrieve JSON data via URL every X second and store in Amazon DynamoDB

1 Answers1

Linked