4

I am trying to use rsync to sync my S3 bucket with my EC2 servers. However I am having trouble coming up with the code. On my EC2 server I have tried the following, but it doesn't work. I know my S3 address is wrong but I'm not sure what to put in its place. iosSourceCode is the bucket name. How can I sync the files in this bucket to my EC2 server's files? After I get this to work I was going to set up a cronjob to do this every 10 minutes or whatever. Is there a better way to do this and if so how? Please provide code, thanks!

sudo rsync -ra iosSourceCode.s3-website-us-east-1.amazonaws.com /var/www/
Blane Townsend
  • 2,888
  • 5
  • 41
  • 55

3 Answers3

7

You can use this command:

aws s3 --region <your region name> sync s3://<your bucket name> /your/directory/path

So in your case:

aws s3 --region us-east-1 sync s3://osSourceCode.s3-website /var/www/

This is a one-way sync (only downloads updates from S3 in this example) so if you want to sync both ways then you need two commands with the source and destination swapped around:

aws s3 --region us-east-1 sync s3://osSourceCode.s3-website /var/www/
aws s3 --region us-east-1 sync /var/www s3://osSourceCode.s3-website

You can add this to a crontab entry to make it occur periodically, as per the following example:

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin
MAILTO=root
HOME=/

*/2 * * * * root aws s3 sync --region ap-southeast-2 /var/www/html s3://mywebsitecode/ >> /usr/local/logs/aws.log 2>&1
*/5 * * * * root aws s3 sync --region ap-southeast-2 s3://mywebsitecode/ /var/www/html >> /usr/local/logs/aws.log 2>&1
brendan
  • 290
  • 2
  • 13
  • 1
    This should be the accepted answer, as it's part of the standard AWS CLI tool, and by all accounts is faster too. See [docs](http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html) and [related answer](http://stackoverflow.com/questions/8659382/downloading-an-entire-s3-bucket/18762381#18762381). – RichVel Feb 05 '17 at 09:58
2

Please use s3cmd

http://s3tools.org/s3cmd

use s3cmd sync

syntax will be as below

s3cmd sync s3://mybucket/myfolder/files/ /var/mybucket/myfolder/files/

You can put above syntax in shell script and add script to cron to run it as specific time interval.

Sandesh Deshmane
  • 2,247
  • 1
  • 22
  • 25
0

You might want to have a look at the s3 sync command that is part of the Amazon CLI (Command line interface).

vjones
  • 386
  • 2
  • 12