Normalizing Disparity Image Using an Accelarator

Question

I have a disparity image and I am normalizing it using the sample code below but it is very slow. I need to do it using some accelerator like custom CIFilter or any other technique but I dont know how? I am currently running the code with CIContext() and it is running on CPU(not sure). Is there a way to run it on GPU and accelerate without custom CIfilter? Here is the current code:

extension CVPixelBuffer {

  func normalize() {

    let width = CVPixelBufferGetWidth(self)
    let height = CVPixelBufferGetHeight(self)

    CVPixelBufferLockBaseAddress(self, CVPixelBufferLockFlags(rawValue: 0))

    let baseAddr = CVPixelBufferGetBaseAddress(self)!
    let floatBuffer = unsafeBitCast(CVPixelBufferGetBaseAddress(self), to: UnsafeMutablePointer<Float>.self)

    var minPixel: Float = 1.0
    var maxPixel: Float = 0.0

    for y in 0 ..< height {
      for x in 0 ..< width {
        let pixel = floatBuffer[y * width + x]
        minPixel = min(pixel, minPixel)
        maxPixel = max(pixel, maxPixel)
      }
    }


    let range = maxPixel - minPixel

    for y in 0 ..< height {
      for x in 0 ..< width {
        let pixel = floatBuffer[y * width + x]
        floatBuffer[y * width + x] = (pixel - minPixel) / range
      }
    }

    CVPixelBufferUnlockBaseAddress(self, CVPixelBufferLockFlags(rawValue: 0))
  }
}

score 1 · Answer 1 · answered Nov 14 '19 at 20:01

1

For your use case, vImage is probably the best option. See Option 3 from this answer.

There are also ways to do that in Core Image. I would imagine using the CIAreaMinMax filter for getting the extrema and then using some clever blending for normalization. If you want I can elaborate on that.

answered Nov 14 '19 at 20:01

Frank Rupprecht

9,191
31
56

thank you for your answer. Yes, would be very usefull if you give code example. – eral Nov 15 '19 at 07:37
1

I tried to get it working with built-in CI filters but no luck so far. Using `vImage` or `vDSP` is most likely the faster option for your use case anyways. – Frank Rupprecht Nov 18 '19 at 07:31
Thank you. I will. – eral Nov 19 '19 at 17:48

Flex Monkey · Accepted Answer · 2019-11-15T17:06:13.813

1

You have the pixel values as Float values, so you could also use vDSP.

vDSP_minv and vDSP_maxv compute the extrema, and:

floatBuffer[y * width + x] = (pixel - minPixel) / range

Can be replaced by vDSP_vasm (you'll need to multiply by the reciprocal of range).

It might also be useful to look at vDSP_normalize which does this calculation:

            m = sum(A[n], 0 <= n < N) / N;
            d = sqrt(sum(A[n]**2, 0 <= n < N) / N - m**2);

            if (C)
            {
                // Normalize.
                for (n = 0; n < N; ++n)
                    C[n] = (A[n] - m) / d;
            }

edited Nov 15 '19 at 17:06

answered Nov 15 '19 at 01:29

Flex Monkey

3,583
17
19

I am taking this as a correct answer although no needed it anymore(I runned my code on real device with CIContext and it is fast enough). I will use your code if I need extra speed in the future. Thank you again! – eral Nov 19 '19 at 17:47

score 0 · Answer 3 · answered Oct 12 '20 at 21:12

I used the Accelerate Framework vDSP vector functions to normalize disparity. See a modified PhotoBrowse in gitHub for a working demo.

Here's the relevant code in two functions

extension CVPixelBuffer {
    func vectorNormalize( targetVector: UnsafeMutableBufferPointer<Float>) -> [Float] {
        // range = max - min
        // normalized to 0..1 is (pixel - minPixel) / range

        // see Documentation "Using vDSP for Vector-based Arithmetic" in vDSP under system "Accelerate" documentation

        // see also the Accelerate documentation section 'Vector extrema calculation'
        // Maximium static func maximum<U>(U) -> Float
        //      Returns the maximum element of a single-precision vector.

        //static func minimum<U>(U) -> Float
        //      Returns the minimum element of a single-precision vector.


        let maxValue = vDSP.maximum(targetVector)
        let minValue = vDSP.minimum(targetVector)

        let range = maxValue - minValue
        let negMinValue = -minValue

        let subtractVector = vDSP.add(negMinValue, targetVector)
            // adding negative value is subtracting
        let result = vDSP.divide(subtractVector, range)

        return result
    }

    func setUpNormalize() -> CVPixelBuffer {
        // grayscale buffer float32 ie Float
        // return normalized CVPixelBuffer

        CVPixelBufferLockBaseAddress(self,
                                     CVPixelBufferLockFlags(rawValue: 0))
        let width = CVPixelBufferGetWidthOfPlane(self, 0)
        let height = CVPixelBufferGetHeightOfPlane(self, 0)
        let count = width * height

        let bufferBaseAddress = CVPixelBufferGetBaseAddressOfPlane(self, 0)
            // UnsafeMutableRawPointer

        let pixelBufferBase  = unsafeBitCast(bufferBaseAddress, to: UnsafeMutablePointer<Float>.self)

        let depthCopy  =   UnsafeMutablePointer<Float>.allocate(capacity: count)
        depthCopy.initialize(from: pixelBufferBase, count: count)
        let depthCopyBuffer = UnsafeMutableBufferPointer<Float>(start: depthCopy, count: count)

        let normalizedDisparity = vectorNormalize(targetVector: depthCopyBuffer)

        pixelBufferBase.initialize(from: normalizedDisparity, count: count)
            // copy back the normalized map into the CVPixelBuffer

        depthCopy.deallocate()
//        depthCopyBuffer.deallocate()

        CVPixelBufferUnlockBaseAddress(self, CVPixelBufferLockFlags(rawValue: 0))

        return self

    }

}

Normalizing Disparity Image Using an Accelarator

3 Answers3