ios Vision VNImageRequestHandler orientation issue

Question

I am trying to detect faces via camera using VNImageRequestHandler (iOS Vision). When I point on the photo by the camera in landscape mode it detects faces but with opposite orientation mode.

  let detectFaceRequestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:])

score 6 · Answer 1 · answered Jun 26 '18 at 17:47

Have you tried to play with the VNImageRequestHandler orientation property?

let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: .right, options: [:])

I had to set it to .right while reading video input from back camera in portrait mode.

Guna pandian · Answer 2 · 2017-06-27T10:27:20.637

1

Convert the image to CIImage and apply the orientation property like below and pass it to the imagerequest handler

let orientation = CGImagePropertyOrientation(uiImage.imageOrientation)
   let imageElement = ciImage.applyingOrientation(Int32(orientation.rawValue))

        // Show the image in the UI.
        originalImage.image = uiImage

also check https://github.com/gunapandianraj/iOS-Vision code for converting Vision rect to UIKit rect

edited Jun 27 '17 at 10:27

answered Jun 27 '17 at 10:07

Guna pandian

83
1
9

on trying to work with the above code for a I get `Use of unresolved identifier 'uiImage'`as soon as I attempt to declare the variable `orientation` in the first line. Is there a specific library that needs to be imported for this to work? – Veejay Mar 22 '18 at 20:35
sorry for the naming convention . uiImage is the local variable of type UIImage which will be assigned with image i picked from photo library or taking photo or downloaded image ... – Guna pandian Mar 24 '18 at 14:30
what is imageElement? and where is used? – Roberto Ferraz Jun 25 '18 at 13:29

sabiland · Answer 3 · 2022-03-16T15:31:48.340

I always got flipped points (bounding boxes) vertically. I fixed this totally with helper method:

private static func translateVisionToNormalBoundingBox(bb: CGRect, imageFullRect: CGRect) -> CGRect
{
    let renormalized = VNImageRectForNormalizedRect(bb, Int(imageFullRect.width), Int(imageFullRect.height))
    // Vertically translate origin !!!
    // Vertically translate origin !!!
    // Vertically translate origin !!!
    return CGRect(
        origin: CGPoint(
            x: renormalized.origin.x,
            y: imageFullRect.maxY - renormalized.origin.y - renormalized.size.height
        ),
        size: renormalized.size
    )
}

For cgImage orientation I use this StackOverflow's extension:

extension UIImage {

var cgImagePropertyOrientation: CGImagePropertyOrientation {
    switch imageOrientation
    {
    case .down: return .down
    case .left: return .left
    case .right: return .right
    case .up: return .up
    case .downMirrored: return .downMirrored
    case .leftMirrored: return .leftMirrored
    case .rightMirrored: return .rightMirrored
    case .upMirrored: return .upMirrored
        
        // TWEAK FOR NEW CASES !!!
        // TWEAK FOR NEW CASES !!!
        // TWEAK FOR NEW CASES !!!
    @unknown default:
        return .down
    }
}

}

ios Vision VNImageRequestHandler orientation issue

3 Answers3

Linked