5

Im developing an application in Qt which, at some point, process a bunch of videos. It works fine but it had only a 40-60% of the cpu usage during the process phase so i tried to make it multithreaded.

I used QtConcurrent cause his 'high leveness' instead a more traditional thread management, my code is simply:

for(int i = 0; i < totalVideos; i++)
{
    QFuture<ResultClass *> futureToken = QtConcurrent::run(this, process, listOfVideos.takeFirst());
    QFutureWatcher<ResultClass *>* fw = new QFutureWatcher<ResultClass *>();
    connect(fw, SIGNAL(finished()), this, SLOT(manageResult));
    fw->setFuture(futureToken);
}

aaaand it works, 100% cpu usage and its around 25-30% faster. But it spawns around 65 new threads (regardless it process 25 or 250 videos) and most of those threads doesn't disappear after the process phase.

My question is: Is this approach right? Is it too raw? Should i control 'manually' the thread creation? Does the QtConcurrent module takes care of all so i should not care of the thread management? Are 85 threads too much? Should i try to kill some of those after the process phase??

Every observation has been done just looking at the Activity monitor.

Thanks in advance.

Sommerwild
  • 506
  • 1
  • 6
  • 18
  • 2
    another option is using a QThreadPool and limit the number of threads directly (also your futureWatcher leaks, unless you delete it in the manageResult slot) – ratchet freak Nov 14 '13 at 09:02
  • Yes, i delete them (with deleteLater())in the slot, thanks for pointing that out. – Sommerwild Nov 14 '13 at 10:31
  • technically they still leak when `this` gets destroyed before the future is finished (add this to the constructor to fix that) – ratchet freak Nov 14 '13 at 11:08

1 Answers1

6

The future of QtConcurrent seems to be uncertain if you read this thread.

Having more threads that processing cores is somewhat redundant. If you have one core and 2 threads running, the processor spends time switching between the processing the 2 threads, but gives the user the appearance of simultaneous processing.

With the same number of cores and threads, the threads can be split between the cores.

Once you have more cores than threads, you're back to the original method of cores jumping up and back between the threads that it is required to process.

Using QThread is actually very easy to do as QThread is not directly a thread, but a thread controller. You can read about how to 'really truly use QThreads' here.

As it describes, you create objects inherited to QObject and move that to a QThread. What is rarely mentioned is that you can move multiple objects to the new QThread, if required.

TheDarkKnight
  • 27,181
  • 6
  • 55
  • 85
  • I know its redundant but i didn't create them. i though that QtConcurrent moves the task to an apropiate threads it says in the docs: "Runs function in a separate thread. The thread is taken from the global QThreadPool. Note that the function may not run immediately; the function will only be run when a thread is available." I dont know where all that thread spawning are coming from. – Sommerwild Nov 14 '13 at 10:24
  • It may be that whatever is doing the video processing is also spawning threads. I suggest converting your code to use QThread and see what performance you get out of it as a comparison. – TheDarkKnight Nov 14 '13 at 11:01
  • 1
    I'll do that, and will post the results. – Sommerwild Nov 14 '13 at 11:11
  • 1
    "Having more threads that [sic] processing cores is somewhat redundant" -- only if the parallel tasks are compute-only; if the tasks have any I/O operations then more threads may be desirable to prevent cores from being idle when they could be processing. – boycy Jul 15 '14 at 10:45