I read about efficiency in parallel computing, but never got an clear idea about it, also I read about achieving efficiency >1 and conclude that it's possible when it's a super linear.
Is that correct and possible?
If yes, then can anybody tell me how and provide an example for that?
Or, if it is not, then why?
A set of processes may get scheduled for execution under several different strategies --
[SERIAL] - i.e. execute one after another has finished, till all are done, or[PARALLEL] - i.e. all-start at once, all-execute at once, all-terminate at onceor
[CONCURRENT] fashion - i.e. some start at once, as resources permit, others are scheduled for [CONCURRENT] execution whenever free or new resources permit. The processing gets finished progressively, but without any coordination, just as resources-mapping and priorities permit.Given an efficiency may be related to power-consumption or to processing-time, let's focus on processing-time, ok?
Gene Amdahl has elaborated domain of generic processing speedups, from which we will borrow here. The common issue in HPC / computer science education is that lecturers do not emphasise the real-world costs of organising the parallel-processing. For this reason, the overhead-naive ( original ) formulation of the Amdahl's Law ought be always used in an overhead-strict re-formulation, because otherwise any naive-form figures are in parallel-computing just comparing apples to oranges.
On the other hand, once both the process-add-on-setup-overhead costs and process-termination-add-on-overhead costs are recorded into the scheme, the overhead-strict speedup comparison starts to make sense to speak about processing-time efficiency.
Having said this, there are cases, when processing-time efficiency can become > 1, while it is fair to say, that a professional due care has to be taken and that not all processing-types permit to gain any remarkable speedup on whatever large-scale pool of code-execution resources, right due to the obligation to pay and cover the add-on costs of the NUMA / distributed-processing overheads.
with an overhead-strict Amdahl Law re-formulation [PARALLEL]-processing speedups GUI-interactive-tool cited here
Being in Sweden, your must-read is the Andreas Olofsson's personal story about his remarkable effort and experience with piloting parallel-hardware with many first-ever-s on the way from Kickstarter to a DARPA-acquired [PARALLEL]-hardware know-how.