0

I am working with kinect in openframework using the ofxKinect addon, which is great and plenty fun!

Anyway I am looking for some pointers or a direction when dealing with multiple bodies on the screen. I was thinking of making a rect around each detected body and when the rects intersect something could happen, an effect or anything.

So what I am looking for are ideas or something that could point me to the right direction of detecting multiple bodies when using a kinect.

Right now based on the depth image I get from the kinect I go through each pixel and create a bunch of smaller rectangles with a padding and group them in a larger rectangle bound if they are separate from another rectangle group. This is not ideal as it only deals with the pixel values and is not really seperating bodies from eachother and is not giving me the results I am looking for.

So any ideas would be greatly appreciated!

Placeable
  • 589
  • 8
  • 22

1 Answers1

1

If you want to use ofxKinect a quick solution would be to threshold on depth and assume bodies and no other objects will be within a depth range. This should make it easy to use the OpenCV's contour finder to isolate the outlines of the bodies and get the bounding rectangles. If the rectangles intersect(and ofRectangle already does the math you), trigger the reaction you need. Also don't forget to do that once if the effect isn't showing already, otherwise you will trigger the effect multiple times per second while the two bodies' bounding rectangles intersect.

You could try something a bit more hardcore and using ofxCv(not just ofxOpenCV) to tap into the HoG functionality. This is slow in itself and not ideal with the depth map, but hopefully you can run in every few seconds just to detect a person and the depth, then keep tracking that movement.

Personally, if you want to track people with the Kinect I recommend using ofxOpenNI as if already provides the scene segmentation feature and even if you don't track the skeletons you can still get useful information like the pixels pertaining to each body and they're centre of mass. I'm guessing Microsoft KinectSDK has a similar feature and there should be an oF addon, but it's windows only.

ofxKinect/libfreenect does not offer any people detection features, so you will need to roll your own.

George Profenza
  • 50,687
  • 19
  • 144
  • 218
  • Thanks for this great answer! Exactly what I was looking for. Didn't realize there was a contour finder so I will look into that one first. I would give another +1 for the links :) As for ofxOpenNI I have it in my addons folder but feel it is a bit annoying to get it integrated into my current project but I will look into that one as well. I believe the first approach with the ofxKinect should be sufficient. – Placeable May 21 '13 at 07:24
  • Just wanted to give an small update. The contour finder works great! Exactly what I was looking for so much obliged! In a way it was much like my own approach just... better and more easy to use :) – Placeable May 21 '13 at 08:01
  • Cool! Glad you hear you're on track :) Share a link later on to your project if you get a chance. – George Profenza May 21 '13 at 09:08
  • I had to make a small adjustment to the findContours function of the ofxCvContourFinder though. It would "merge" the blobs if they hit eachother but I need them seperate for the intersection test. This was fixed with a distance check of the blobs X & Y center pixel value on the depth image. So only blobs with a distance really close to each other are merged into a larger blob (Where I would assume it is the same body) any other blobs outside this distance is treated as separate blobs and will not merge to a larger one. This way bodies on different distance from the kinect will be isolated. – Placeable May 21 '13 at 09:33