How to determine if a face is flat like an image using ARKit?

Question

How to determine if a face detected is flat like an image or has a varying topography in ARKit? I'm currently using ARFaceTrackingConfiguration.

One approach I tried was finding the average distance between each vertex from the face geometry and if it's below a certain arbitrary threshold, then deem the face as flat.

let vertices = faceGeometry.vertices
var totalDistance: Float = 0
for i in 0..<vertices.count {
    let vertex = vertices[i]
    totalDistance += simd_length(vertex)
}
let averageDistance: Float = totalDistance / Float(vertices.count)
if averageDistance < 0.07 {
    print("Flat face")
} else {
    print("Real face")
}

The idea was that a flat face would generally have closer vertices on average. However, this method provided mixed results upon testing.

I've also tried comparing the depth of a fake face (an image) and a real 3D face, but for reason they turned out to be the same when the former should be flat.

var minZ = Float.infinity
var maxZ = -Float.infinity
for vertex in faceAnchor.geometry.vertices {
    let vertexVector = simd_float4(vertex, 1)
    let worldVertex = faceAnchor.transform * vertexVector
    let transformedVertex = SCNVector3(worldVertex.x, worldVertex.y, worldVertex.z)
    minZ = min(minZ, transformedVertex.z)
    maxZ = max(maxZ, transformedVertex.z)
}

let depth = maxZ - minZ
print(depth)

Answer 1

AVFoundation + Vision

You need to analyze the depth map of a flat image with a face on it (or a person with a real topology) for the uniformity of a grey color. A flat image has a subtle gradient (almost solid grey color), while a real human face has a significantly large number of grey tones. To optimize Apple Vision analysis for detected face, use VNDetectFaceRectanglesRequest class to restrict processing within a bounding box.

Here's a sample code helping you visualize a depth data coming from the TrueDepth camera.