1

I have a program in C, which takes arbitrary number of files as commandline argument, and calculates sha1sum for every file. I am using pthreads, so that I can take advantage of all 4 my cores.

Currently, my code runs all threads in parallel at the same time. Here is a snippet:

c = 0;
for (n = optind; n < argc; n++) {
    if (pthread_create(&t[c], NULL, &sha1sum, (void *) argv[n])) {
        fprintf(stderr, "Error creating thread\n");
        return 1;
    }
    c++;
}

c = 0;
for (n = optind; n < argc; n++) {
    pthread_join(t[c], NULL);
    c++;
}

Obviously, it is not efficient (or scalable) to start all threads at once.

What would be the best way to make sure, that only 4 threads are running at any time? Somehow I need to start 4 threads at the beginning, and then "replace" a thread with new one as soon as it completes.

How can I do that ?

Martin Vegter
  • 136
  • 9
  • 32
  • 56

2 Answers2

2

Obviously, it is not efficient (or scalable) to start all threads at once.

Creating 4 threads is not necessarily provides the best performance on a 4 core machine. If the threads are doing IO or waiting on something, then creating more than 4 threads could also result in better performance/efficiency. You just need to figure out an approximate number based on the work your threads do and possbily a mini-benchmark.

Regardless of what number you choose (i.e. number of threads), what you are looking for is a thread pool. The idea is to create a fixed number of threads and feed them work as soon as they complete.

See C: What's the way to make a poolthread with pthreads? for a simple skeleton. This repo also shows a self-contained example (check the license if you are going to use it). You can find many similar examples online.

Community
  • 1
  • 1
P.P
  • 117,907
  • 20
  • 175
  • 238
0

The thing you are looking for is a semaphore; it will allow you to restrict only 4 threads to be running at a time. You would/could start them all up initially, and it will take care of letting a new one proceed when a running one finishes.

Scott Hunter
  • 48,888
  • 12
  • 60
  • 101