Stack Overflow
Stack Overflow

Questions tagged [aws-datasync]

40 questions
14
votes
1 answer

AWS data sync vs storage gateway use case

What is the real world use case difference between AWS data sync and storage gateway, since both allow data transfer from/to on premise data centre and AWS.
aman
  • 278
  • 3
  • 11
11
votes
2 answers

AWS DataSync vs. AWS Transfer Family

I read the documentation from the official website. but it does not give me a clear picture. Why would need to use AWS Transfer Family since AWS DataSync can also achieve the same result? I notice the protocol differences, but am not quite sure…
Turbo
  • 548
  • 5
  • 13
6
votes
6 answers

AWS Datasync S3 -> S3 cross account, confused about destination role/account

I want to use Datasync to copy data from a single S3 bucket in one account to a single S3 bucket in another account. I'm following this official AWS Datasync blog:…
Alex Kerr
  • 956
  • 15
  • 44
4
votes
1 answer

How to grant EFS mount target access to DataSync Agent on-premise?

We have an on-premise DataSync agent (VM image) running, and an EFS with mount target. We want to grant the agent access to the mount target in order to run sync tasks. However, there does not seem to be any security group assignable to the agent…
Richard Kiefer
  • 1,814
  • 2
  • 23
  • 42
3
votes
1 answer

AWS DataSync could not detect any files in the source NFS filesystem

I am trying to Sync the data from a On Premise VM to AWS S3 bucket using AWS Data Sync, I have already configured the AWS Data Sync Agent on the On Prem VM , The Agent is now Online and we have also created a new task, The task is available in…
Ajinkya
  • 843
  • 10
  • 32
2
votes
1 answer

AWS Autosync : moving data from FTP server to S3

I have a workflow which moves 700gb in files from an ftp server to an on-prem server for python script processing. i would like to migrate this process to a AWS s3 bucket for lambda to process. i saw AWS data-sync as a reasonable priced solution…
phill
  • 13,434
  • 38
  • 105
  • 141
2
votes
3 answers

Need to transfer data from AWS EFS to EFS in different regions

I have an EFS configured in the oregon region but as the EFS is now available in the Tokyo region so I want to transfer all EFS data to the Tokyo region where EFS is already created. AWS doc mentioned AWS-DataSync but it does not work as expected…
Umar Hayat Khan
  • 29
  • 1
  • 6
1
vote
1 answer

Permission issue when creating AWS datasync task

I am trying to use AWS datasync service to copy files from one bucket in one account to another bucket in another account. Also the buckets are in a different region. I get this error message when I create the Datasync task: Failed to create…
zenc
  • 31
  • 3
1
vote
1 answer

aws datasync cause double trigger for lambda event

I use AWS DataSync service to sync two buckets A and B The DataSync defined to check integrity during transfer and copy only new data to another bucket (B). I have lambda function that defined to trigger on new files from bucket B All files that…
Ziv M
  • 409
  • 6
  • 18
1
vote
1 answer

ECS Fargate with EFS Volume update data

I have an ECS Service with Tasks using an EFS Volume. I also have a Datasync task to update the files on the EFS Filesystem(destination) from an S3 Bucket(source) The container task is serving files stored on the EFS. Whenever I run the Datasync…
arbermejo
  • 106
  • 5
1
vote
0 answers

How can I install AWS Data sync agent on Hadoop cluster?

I have my hadoop sandbox set up on a Virtual Machine on Azure Cloud. I would like to install aws data sync agent on my hadoop sandbox so that I can create a task for data copy from my Hadoop cluster to AWS S3 using AWS Data Sync. Can someone guide…
pri
  • 23
  • 3
1
vote
1 answer

AWS DataSync from EFS to S3 - connection timed out

I'm trying to create a DataSync task to copy files from EFS to S3, and for this I'm using Terraform. From reading the documentation, it looks like I dont need DataSync agent to do this. Following the guide at…
caffeine_inquisitor
  • 617
  • 1
  • 9
  • 21
1
vote
1 answer

How to sync RDS database across regions in AWS

I have two MYSQL RDS Instance in Ohio and Mumbai region. Both RDS instance should sync each other. Does AWS have any services to enable sync between RDS Database? else should we configure to achieve? Both RDS should sync by hourly or daily. Can any…
Lakshminarayanan S
  • 85
  • 1
  • 14
1
vote
1 answer

Describing ENI error when executing DataSync task in Fargate/Lambda

I am experiencing the error "InvalidRequestException: Invalid request made for describing ENI" when executing AWS DataSync task from Fargate/Lambda. What could be the cause of this?
Jiji
  • 1,092
  • 3
  • 15
  • 36
1
vote
2 answers

Migrate existing data from Ali Cloud OSS to Amazon S3 Bucket

I am wondering if there is any AWS service to migrate existing data from ali cloud OSS to Aws S3 bucket. Here is a service in Ali Cloud which migrates from S3 to OSS. I need…
Sheharyar Naseem
  • 71
  • 5
1
2 3