design pattern(s) or best practice for image acquisition and image processing in real time application

Question

First, by "real-time", image processing of an image should take 0.1 second or less in this application.

In our application, three threads in addition to main thread are running. One for image acquisition, second for image processing, third for robot. Between the two threads, there is an image queue to share so while camera en-queues images and robot de-queues the processed images, imaging processor de-queues images and enqueues processed images. One restriction you might've noticed is that processed images should be in sequence meaning keeping the same order of images as in image acquisition.

Is there any design pattern or best practice to be applied for this architecture.

Sounds reasonable. Maybe you should do some profiling to identify bottlenecks if you have a performance issue? If it's the image-processing, you could maybe add another thread or two, but then you're probably going to need sequence-numbers in your image class to ensure sequential O/P after processing. Where does the O/P go? — Martin James, Aug 18 '12 at 14:03
@UkuLoskit Thanks for the pointer but I think we are doing it already with image queue sync. — Tae-Sung Shin, Aug 18 '12 at 14:08
@MartinJames in our case, processed/altered image goes to robot. So yes, I think keeping sequence would be important. But having thread for each image processing would be an good option, I think. Thanks for your comment — Tae-Sung Shin, Aug 18 '12 at 14:09

Tanner Sansbury · Accepted Answer · 2012-08-20T17:46:49.837

The pipes and filter pattern can be well suited for this.

The acquisition filter needs to be serial in order.
The processing filter can run in parallel.
The transport-to-robot filter needs to be serial in order.

In order to accomplish this with pre-existing technology, I have seen realtime applications processing large amounts of data use Intel's Threading Building Blocks (TBB). In the Thread Building Blocks Tutorial, the "Working on the Assembly Line: pipeline" section describes a similar problem:

A simple text processing example will be used to demonstrate the usage of pipeline and filter to perform parallel formatting. The example reads a text file, squares each decimal numeral in the text, and writes the modified text to a new file. [...] Assume that the raw file I/O is sequential. The squaring filter can be done in parallel. That is, if you can serially read n chunks very quickly, you can transform each of the n chunks in parallel, as long as they are written in the proper order to the output file.

And the accompanying code:

void RunPipeline( int ntoken, FILE* input_file, FILE* output_file ) {
  tbb::parallel_pipeline(
    ntoken, 
    tbb::make_filter<void,TextSlice*>( 
      tbb::filter::serial_in_order, MyInputFunc(input_file) )
  & tbb::make_filter<TextSlice*,TextSlice*>( 
      tbb::filter::parallel, MyTransformFunc() )
  & tbb::make_filter<TextSlice*,void>(
      tbb::filter::serial_in_order, MyOutputFunc(output_file) ) );
}

Regardless of whether or not TBB is used, it can serve as a great implementation reference for a pipe and filter pattern that decouples the pattern from the algorithms, while providing the ability to control data order/threading for the filters.

score 1 · Answer 2 · answered Aug 18 '12 at 14:10

1

I think your approach is correct.

The possible improvement could be usage of thread pools, for example for image processing (especially if it takes much more time than acquisition). You could consider OpenMP or boost threadpool, or boost::asio::io_service

answered Aug 18 '12 at 14:10

nogard

9,432
6
33
53

I think I know what thread pool is. But my question is how the thread pool can be used in this architecture. – Tae-Sung Shin Aug 18 '12 at 14:17
@david Instead of one processing thread you will have a group of threads reading images from the queue and processing them in parallel. The actual implementation depends on your needs – nogard Aug 18 '12 at 14:17
Got it but one restriction is that the image processed should be in sequence for the next stage. I guess I need to edit my question. – Tae-Sung Shin Aug 18 '12 at 14:20
Still possible to use threadpool with the following scenario for processing thread: it takes not one, but NUMBER_OF_CORES images from the queue and passes them to the threadpool executor which processes this bunch in parallel. The sequence will be preserved in this case – nogard Aug 18 '12 at 14:24
OK. It makes sense. program can remember the order. Thanks – Tae-Sung Shin Aug 18 '12 at 14:29

design pattern(s) or best practice for image acquisition and image processing in real time application

2 Answers2