aws s3 sync vs rclone vs something else?

Question

I need upload my incremental backups ~250 tb to my aws s3 bucket.

AWS CLI is pretty tool for me. But I would like to receive substantiated reasons. — Diamon, Sep 10 '19 at 07:22
afaik aws cli doesn't let you sync buckets with 2 different sets of credentials. but rclone does see https://stackoverflow.com/a/69517798/3140992 — citynorman, Oct 10 '21 at 18:18

score 0 · Answer 1 · answered Sep 13 '22 at 10:27

You can use S3CMD to copy data to S3 from local machine or any Server with terminal access

On Ubuntu

$ sudo apt-get install s3cmd

Install Latest s3cmd using Source

$ wget https://sourceforge.net/projects/s3tools/files/s3cmd/2.2.0/s3cmd-2.2.0.tar.gz

$ tar xzf s3cmd-2.2.0.tar.gz

To install s3cmd

$ cd s3cmd-2.2.0 
$ sudo python setup.py install

configure s3cmd using the Accesskey and Secret key Generated from AWS Credentials

$ s3cmd --configure

List complete S3 Bucket

$ s3cmd ls

Creating a S3 Bucket

$ s3cmd mb s3://bucket_name

Upload a file in to Bucket

$ s3cmd put file.txt s3://bucket_name/

Upload a complete directory in S3 Bucket

$ s3cmd put -r backups s3://bucket_name/

Upload a complete directory in a S3 Bucket containing Folder

$ s3cmd put -r backups s3://bucket_name/folder_name/

Automatically Upload Backups in s3 using cronjob

Create a Cronjob

$ crontab -e

add below cmd and save the file

0 1 * * * s3cmd put -r backups s3://bucket_name/folder_name/backups_$(date +"%F %T")

1 Answers1