3

I'm trying to use the Accelerate framework on iOS to bypass the fact that Core Image on iOS doesn't support custom filters/kernels. I'm developing an edge detection filter using two convolutions with a Sobel kernel, but starting with a simple Gaussian blur to get the hangs of it. I know vImage is geared towards image manipulation as matrices and vDSP focuses in processing digital signals using Fourier transforms. But although I started using the vImage functions (vImageConvolve_XXXX, etc), I'm hearing a lot of people discussing the use of vDSP's functions (vDSP_conv, vDSP_imgfir, etc) to do such things as convolutions. So that leads me to the question at hand: when should I use one over the other? What are the differences between them with regards to convolution operations? I've looked everywhere but couldn't find a clear answer. Can someone shed some lights on it, or point me in the right direction?

Thanks!

Stephen Canon
  • 103,815
  • 19
  • 183
  • 269
mgaldieri
  • 143
  • 1
  • 7
  • 1
    As an FYI, I have a very fast GPU-accelerated Sobel edge detection filter as part of my [GPUImage](https://github.com/BradLarson/GPUImage) framework. It can process a 640x480 frame of video in 2.5 ms on an iPhone 4, which looks to be far faster than even Accelerate-based convolution. – Brad Larson Mar 20 '12 at 17:11
  • @Brad-Larson Wow! This one looks very promising! It's almost like OpenCL for iOS:) Thanks for the framework and for the benchmarks. I suspected the OpenGL route would yield better results performance-wise. But due to memory limitations (and the consequent image size constraints) CPU processing (via Accelerate, CoreImage, and the likes) is still the preferred way to manipulate large images, eg. stored in the image Library (since they're probably larger than 2048x2048 pixels), even thou it's slower. I'll take it for a spin as soon as I brush up my GLSL skills! – mgaldieri Mar 21 '12 at 23:12
  • 1
    The A5-based devices (iPhone 4S, iPad 2, Retina iPad) all support 4096x4096 textures, but you're right that we need some kind of tiling implementation to handle these larger images. I have an idea of how I might do that, but it will take a little work. – Brad Larson Mar 21 '12 at 23:17

2 Answers2

2

If vImage provides the operation you need, it is usually simplest to use that. vImage does cache blocking and threading for you, vDSP does not. vImage provides operations on interleaved and integer formats, which are often useful for image processing.

Stephen Canon
  • 103,815
  • 19
  • 183
  • 269
  • Thanks Stephen! During my testings, I've found that vImage also can run a bit faster than the same algorithm using vDSP (with regards to matrix transformations & convolutions). Also someone pointed my at another forum that vImage wasn't available on iOS prior to version 5! – mgaldieri Jan 29 '12 at 04:04
2

Last time I experimented, neither of these frameworks took advantage of kernel separability, which affords a huge performance boost when convolving in the spatial domain -- a far larger performance boost than vectorized instructions will ever buy you. The Sobel kernel in particular is separable, so if you're using vDSP or vImage (instead of say OpenCV), be sure to separate the kernel yourself.

Josh Bleecher Snyder
  • 8,262
  • 3
  • 35
  • 37
  • If you think it would be useful for vImage to automatically detect kernel separability, please file a bug report to request that feature. – Stephen Canon Jan 29 '12 at 17:19
  • 4
    @StephenCanon pretty sure I did. Frankly, it's kinda depressing to file radars, given the effort it takes to write them up, the utter lack of visibility (I'd rather find the duplicate myself and +1 it), and the fact that every single one *always* gets closed as a duplicate. Existential pointlessness is the phrase that comes to mind; I wish it were not so. – Josh Bleecher Snyder Jan 29 '12 at 21:19
  • 1
    Even when they're closed as a duplicate, the are read, and the feedback is very useful. If nothing else, it counts as an extra vote for "this particular problem is important, please give it more resources". – Stephen Canon Jan 29 '12 at 22:26