I tried to migrate data from Google Cloud Platform, I tried this method by Mike Exporting data from Google Cloud Storage to Amazon S3, but got the complaint, anyone knows what should I do next?
Asked
Active
Viewed 599 times
2 Answers
0
Make sure your internet connection is stable, this happens when you don't have a stable network and as a result it will return a broken pipe.
matesio
- 1,584
- 17
- 31
0
Note that when you're using gsutil rsync method, the transferred data will actually go through the machine you run the command from. If it has a poor or unstable internet connection you may face this kind of issues.
To avoid it, one option is to create a Compute Engine VM in your Google Cloud project and run the gsutil rsync command in there, so you can take advantage of Google network. Once the transfer is finished you can delete the VM.
Another option is to use Google Storage Transfer Service.
LundinCast
- 9,412
- 4
- 36
- 48
-
1Note for anyone looking for GCS to S3 transfers. The Transfer Service only writes to GCS destinations. – Devon_C_Miller Jan 11 '22 at 20:48