1

I have tried to copy a .mp4 file from my local directory to my google cloud bucket, using:

gsutil cp my_filefile.mp4 gs://my_bucket

This part works as expected, but when i try to limit the bandwidth, using:

trickle -d 10 -u 10 gsutil cp my_filefile.mp4 gs://my_bucket

the uploading happens at the same rate, and not with 10 kb/s. I have read that trickle does not handle static executable files, which the .mp4 appears to be since when running ldd my_file.mp4, in the terminal, it returns not a dynamic executable.

Has anyone experienced the same issue, and if that is the case, how was the problem handled, or am i approaching this issue the wrong way?

UPDATE 1: Turns out it does not matter what file i use. gsutil still bypasses trickle somehow. I have tested to see if trickle worked with other programs, and it performed as expected, with bandwidth control. I have also tested gsutil mv and gsutil rsync, with the same results, as with cp. I have also tested the bandwidth throttling on an arm64 system, with the same results.

bachla
  • 21
  • 3

1 Answers1

0

You should limit the number of thread and process as described in the documentation. Trickle shouldn't been applied in case of multi process.

trickle -d 10 -u 10 gsutil -o "GSUtil:parallel_process_count=1" \
  -o "GSUtil:parallel_thread_count=1" cp my_filefile.mp4  gs://my_bucket
guillaume blaquiere
  • 66,369
  • 2
  • 47
  • 76
  • I have tried that command from the documentation, and what happens is that a file of size 88 Mb is uploaded to my bucket after approximately 1 second. As far as i understand should it not upload with 10 kb/s when i set `-u 10`? In the terminal it also states: `trickle: Could not reach trickled, working independently: No such file or directory` – bachla Aug 19 '20 at 12:10
  • That's strange. It works on my computer (Windows10 with WSL (ubuntu) installed on it), but doesn't work on Cloud Shell for example.... To remove the error, add a `-s` as option, but it's not the issue I guess... – guillaume blaquiere Aug 19 '20 at 12:44
  • I think i might have to get one of my colleagues to test it, to see if there is a problem with my machine. But thank you for the reply, and tip regarding the error :) – bachla Aug 19 '20 at 13:28
  • Depending on the task it might be beneficial to set the trickle values to 1/10 of what you want, and set process or thread count of gsutil to 10, especially if you're downloading a lot of small files. – Michał Tatarynowicz Aug 29 '21 at 12:14