1

I'm trying to implement a head pose estimation algorithm and I'm using a Time-of-Flight camera. I need to detect the nose tip in the point cloud data I get from the camera. After I know where the nose tip is I would sample N nearest neighbour points around it and do Least Square Error Plane Fitting on that part of the point cloud to retrieve the Yaw and Pitch angles. The nose detection should work for different head poses not just for a full frontal head pose.

I implemented the plane fitting and that works fine but I don't know how to detect the nose tip from the 3D data.

Any advice on how this could be done would be much appreciated.

Regards, V.

vandelfi
  • 89
  • 9
  • I have tried using the haar cascade from OpenCV to detect it in the grayscale image that I also get from the TOF camera. I know the 2D to 3D correspondences so this can also be done. Find the nose tip in a 2D image and retrieve its 3D position. The problem is it only works for frontal poses. – vandelfi Mar 07 '14 at 11:44
  • In 2D I've used geodesics to compare lengths along a curve versus lengths inside a figure: the same should work for 3D point clouds. This could help find localized in-bumps and out-bumps. – Rethunk Mar 07 '14 at 15:25

1 Answers1

2

I used to work with Kinect images that have a limit on depth z > .5m, see below. I hope you don’t have this restriction with your ToF camera. Nose as an object is not very pronounced but probably can be detected using connected components on depth image. You have to find it as a blob on otherwise flat face. You can further confirm that it is a nose by comparing face depth with nose depth and nose position relative to the face. This of course doesn’t apply to the non frontal pose where nose should be found differently.

I suggest inverting your logical chain of processing: find nose then found face and start looking for a head first (as a larger object with possibly better depth contrast) and then for nose. Head is well defined by its size and shape in 3D and a face 2D detection can also fit a raw head model into your 3D point cloud using similarity transform in 3D.

link to Kinect depth map

Community
  • 1
  • 1
Vlad
  • 4,425
  • 1
  • 30
  • 39