OpenMP multiple for loops

Question

I have code that looks something like this:

for(i=0; i<max;i++){
   for(j=0; j<max2;j++)
     //stuff
}

for(i=0; i<max;i++){
   for(j=0; j<max2;j++)
     //other stuff
}

for(i=0; i<max;i++){
   for(j=0; j<max2;j++)
     //final stuff
}

I want to parallelize this using OpenMP. What's the best method? I tried doing the #pragma omp parallel private(i) at the beginning and #pragma omp for before every j loop. This is what I mean:

 #pragma omp parallel private(i)
 {
 for(i=0; i<max;i++){
   #pragma omp for
   for (j=0; j<max2;j++){
   //and so on and so forth

The thing is, this gives me no performance boost whatsoever. I suspect this is because the 3 for loops aren't run in parallel...if I could get those 3 to run at the same time, I think I could get a performance boost. Any ideas? Thanks!

Do you mean that the three loops are not data dependent on each other so they can run in parallel or do you want to distribute each loop among the threads in the OpenMP team? — Hristo Iliev, Nov 27 '12 at 10:01
Yes the three loops are not data dependent on each other so I want them to run in parallel — pauliwago, Nov 27 '12 at 18:06
Why are you parallelizing the inner loop? For parallelization it is generally benefitial to hand each threads a rather big chunk of data, so unless the iterations of the outer loop are dependent on each other (so the loop over `i` must be executed in order) it doesn't really make sense to parallelize the inner loop instead of the outer one. — Grizzly, Nov 27 '12 at 19:55

score 2 · Accepted Answer · edited Jun 30 '18 at 16:57

2

A quick fix is to make a iterative section and parallel this:

#pragma omp for
for (k=0;k<3;k++){
  if (k==0) do_stuff();
  if (k==1) do_other_stuff();
  if (k==2) do_other_other_stuff();
}

A better fix is to use omp sections directive. (Solution taken from here)

#pragma omp parallel sections
{
  #pragma omp section
  {
     /* Executes in thread 1 */
     do_stuff();
  } 
  #pragma omp section
  {
    /* Executes in thread 2 */ 
    do_other_stuff();   
  } 
  #pragma omp section
  {
    /* Executes in thread 3 */
    do_other_other_stuff();
  }   
}

edited Jun 30 '18 at 16:57

Richard

56,349
34
180
251

answered Nov 27 '12 at 11:41

banuj

3,080
28
34

I rarely give -1's to answers on SO and would refrain from doing so in your case too if you educate yourself about the OpenMP `sections` construct and edit your answer accordingly. – Hristo Iliev Nov 27 '12 at 12:32
Sorry, I didn't know about `omp sections`. – banuj Nov 27 '12 at 12:41
Now you know - mission accomplished! :) – Hristo Iliev Nov 27 '12 at 12:53
I thought you can't use `sections` with `for` loops? Am I wrong? – pauliwago Nov 27 '12 at 18:07
@pauliwago, unfortunately yes, you are wrong. OpenMP sections are simply C blocks `{ ... }` and inside those C blocks there could be almost anything, including loops. But loop iterations are not spread among the threads, one has to use a workshare directive for that. – Hristo Iliev Nov 27 '12 at 18:45
So I need to use `#pragma omp section` in addition to `#pragma omp for`. Is that correct? – pauliwago Nov 27 '12 at 18:48
@pauliwago: You'd probably want to use regular for loops, since nesting of openmp worksharing constructs is a bit problematic – Grizzly Nov 27 '12 at 19:57
@pauliwago No. You should use only `#pragma omp section` without `#pragma omp for`. In OpenMP the focus must be on how to parallel big chunks of code instead paralleling smaller portions of code. The bigger task to parallel, the bigger performance you get. – banuj Nov 28 '12 at 08:23
@banuj, I tried running first two examples from https://www.openmp.org/wp-content/uploads/openmp-examples-4.5.0.pdf on my Ubuntu VM (3 vCores) but instead of giving performance boost, 3x to 5x performance degradation is observed. Can you comment on it? – Arshan Jul 05 '18 at 10:27
How many physical cores have you reserved for the vm? If there is only one core or two, than the degradation is normal. – banuj Jul 05 '18 at 11:49
I have reserved 3 CPUs from VirtualBox settings for my VM. – Arshan Jul 06 '18 at 04:50

OpenMP multiple for loops

1 Answers1