-2

Current Scenario : I use process builder to execute Rsync command through Java code to transfer some big file structures from one Linux machine to another. The processor of the machine which executes this Java code(transfer files) is Octa core.

My observation : When I create a task scheduler which executes 8 threads in parallel, I get maximum speed for transfer. (quite convincing, as I am using Octa core processor)

Scenario I want : Now I want to use 32 core processor machine to transfer files. And I use 32 threads in parallel(thread pool of fix size 32) I am asking this question because I donot have 32 core machine and cannot perform test.Adding one more complication, there are 4 machines in parallel which makes total threads 128.The machine where I am transferring data is only one.

Question : Will I get improved performance in terms of transfer speed? Is there any factor considering target machine which will limit my transfer speed?

Parag Vinchurkar
  • 5
  • 3
  • 3
    "The processor of the machine which executes this Java code(transfer files) is 8 bit." are you sure about that? You seem confused about the difference between bits and threads. – Federico klez Culloca Jul 31 '20 at 10:20
  • It looks like you have a lot of confusion here between word size (8 bit/32 bit), cores, and threads. The processor isn't likely to be a bottleneck in any of this, you're going to be limited by network speed and disk read/write long beforehand. – Chris Cooper Jul 31 '20 at 10:21
  • 1
    Usually, disk and network speed are more important than CPU power for file transfers. – Henry Jul 31 '20 at 10:22
  • That was a blunder.... I have edited question – Parag Vinchurkar Jul 31 '20 at 10:22

1 Answers1

0

"When I create a task scheduler which executes 8 threads in parallel, I get maximum speed for transfer...Now I want to use 32 core processor machine to transfer files...Will I get improved performance in terms of transfer speed? Is there any factor considering target machine which will limit my transfer speed?

Improved performance: the main bottle neck is the network bandwidth, not the amount of compute power. However, if rsync is using compression then yes, using more compute (more parallel threads) might help. And in your case, more CPU gave more throughput so let's assume that will happen

Other factors:

Vorsprung
  • 32,923
  • 5
  • 39
  • 63