Averaging the color of pixels with Accelerate

Question

Yes, I know about using CIAreaAverate CIFilter to get the average color of pixels.

I am trying to create some alternative using Accelerate Framework to see if I can come with something faster.

I am rendering a CIImage to a context. For that purpose I have this CIImage extension...

let device: MTLDevice = MTLCreateSystemDefaultDevice()!
let context = CIContext.init(mtlDevice: device, options: [.workingColorSpace: kCFNull])

let w = self.extent.width
let h = self.extent.height
let size = w * h * 4

var bitmap = [UInt8](repeating: 0, count:Int(size))


context.render(self,
                      toBitmap: &bitmap,
                      rowBytes: 4 * Int(w),
                      bounds: self.extent,
                      format: .BGRA8,
                      colorSpace: nil)

At this point I have bitmap containing the BGRA bytes interleaved.

To get the average of R, G and B, all I have to do is something like this:

var averageBlue : Int = 0

for x in stride(from:0, through: bitmap.count-4, by: 4) {
  let value = bitmap[Int(x)]
  averageBlue += Int(value)
}

averageBlue /= numberOfPixels

but this for loop is slow as hell, as expected.

I was thinking about using some Accelerate function like

vDSP_meanvD(bitmap, 2, &r, vDSP_Length(numberOfPixels))

but this function requires bitmap to be an array of UnsafePointer<Double>...

I could convert bitmap to that, but that would require a for loop, that is slow...

Is there any way to extract those R, G and B pixels and have their individual averages using some accelerate stuff going on?

no. Just the average R, average G and average B float values. — Duck, Mar 11 '19 at 16:18
FWIW, I found the old-school, CoreGraphics technique of building a pixel buffer, https://stackoverflow.com/a/31661519/1271826, inspired by this archaic [QA1509](https://developer.apple.com/library/archive/qa/qa1509/_index.html) to be a fraction faster than taking my `UIImage`, building `CIImage`, and using the technique you show above... — Rob, Mar 12 '19 at 04:07

score 2 · Answer 1 · answered Mar 11 '19 at 21:19

2

You can convert bitmap to single-precision floating-point values using vDSP_vfltu8(_:_:_:_:_:) :

let bitmap: [UInt8] = [1, 10,  50,  0,
                       2, 20, 150,  5,
                       3, 30, 250, 10]

//Blue
var blueFloats = [Float](repeating: 0, count: bitmap.count/4)

vDSP_vfltu8(bitmap,
            vDSP_Stride(4),
            &blueFloats,
            vDSP_Stride(1),
            vDSP_Length(blueFloats.count))

And then use vDSP_meanv(_:_:_:_:) :

var blue: Float = 0

vDSP_meanv(blueFloats,
           vDSP_Stride(1),
           &blue,
           vDSP_Length(blueFloats.count))

print("blue =", blue)     //2.0

As to the reds :

//Red
var redFloats = [Float](repeating: 0, count: bitmap.count/4)

vDSP_vfltu8(UnsafePointer.init(bitmap).advanced(by: 2),
            vDSP_Stride(4),
            &redFloats,
            vDSP_Stride(1),
            vDSP_Length(redFloats.count))

var red: Float = 0

vDSP_meanv(redFloats,
           vDSP_Stride(1),
           &red,
           vDSP_Length(redFloats.count))

print("red =", red) //150.0

answered Mar 11 '19 at 21:19

ielyamani

17,807
10
55
90

Be careful with non-unit strides (such as `vDSP_Stride(4)` above). In most cases, vDSP can't vectorise operations with strides that aren't `1`. – Flex Monkey Mar 14 '19 at 13:14
In addition to what @SimonGladman warned about strides, any two pass algorithm (convert, then sum) will necessarily be slower than a halfway-decent single-pass algorithm for this task, so you really want to use API that take input as 8-bit integer. I don't *think* that vImage has an API that does exactly what you want, unfortunately (Simon would know for sure), so the best solution here may be to write a little bit of vector code yourself. – Stephen Canon Aug 15 '19 at 12:51
1

To the best of my knowledge, there are no reduction filters in vImage. _Maybe_ `vImageConvert_ARGB8888toPlanarF` (https://developer.apple.com/documentation/accelerate/1533036-vimageconvert_argb8888toplanarf) will help - that will give you single-precision planar buffers for each color channel. Then you can use `vDSP_meanv` on those and skip `stride`. Note the _maybe_ :) – Flex Monkey Aug 16 '19 at 13:21

score 1 · Accepted Answer · answered Mar 12 '19 at 03:38

Like ielyamani’s said, you can use vDSP_vfltu8 to build that buffer of Float efficiently.

But rather than striding through that array four times, you can also use cblas_sgemv (or cblas_sgemm) to calculate all four averages in a single call:

let pixelCount: Int = width * height
let channelsPerPixel: Int = 4

let m: Int32 = Int32(channelsPerPixel)
let n: Int32 = Int32(pixelCount)
let lda = m

var a = [Float](repeating: 0, count: pixelCount * channelsPerPixel)

vDSP_vfltu8(pixelBuffer, vDSP_Stride(1), &a, vDSP_Stride(1), vDSP_Length(pixelCount * channelsPerPixel))

var x = [Float](repeating: 1 / Float(pixelCount), count: pixelCount)

var y = [Float](repeating: 0, count: channelsPerPixel)

cblas_sgemv(CblasColMajor, CblasNoTrans, m, n, 1, &a, lda, &x, 1, 1, &y, 1)

print(y)

They’re in `y`, in whatever order you had your channels in your pixel buffer. — Rob, Mar 12 '19 at 20:02

Averaging the color of pixels with Accelerate

2 Answers2