5

I'm looking into making a project with the Kinect to allow my Grandma to control her TV without being daunted by using the remote. So, I've been looking into basic gesture recognition. The aim will be to say turn the volume of the TV up by sending the right IR code to the TV when the program detects that the right hand is being "waved."

The problem is, no matter where I look, I can't seem to find a Linux based tutorial which shows how to do something as a result of a gesture. One other thing to note is that I don't need to have any GUI apart from the debug window as this will slow my program down a fair bit.

Does anybody know of something somewhere which will allow me to in a loop, constantly check for some hand gesture and when it does, I can control something, without the need of any GUI at all, and on Linux? :/

I'm happy to go for any language but my experience revolves around Python and C.

Any help will be accepted with great appreciation.

Thanks in advance

Matt

user3046180
  • 195
  • 1
  • 1
  • 5
  • That's pretty cool!(+1). Even though you can implement your own basic gestures using libfreenect and opencv, might be simpler to use the gestures that ship with openni. escapecharacter's suggestion is good. It's also worth taking into account mobility and the fact that some gesture are more exhausting from a motor point of view than others. – George Profenza Nov 28 '13 at 23:15
  • Yes, definitely exhausting gestures is not a good thing, especially for old people. I'm treating the original poster's project as not an "improvement" but a "fun exploration". – escapecharacter Nov 29 '13 at 04:45
  • Me too to be quite honest ;) Any excuse to have some fun with hacking – user3046180 Nov 29 '13 at 14:46

1 Answers1

3

In principle, this concept is great, but the amount of features a remote offers is going to be hard to replicate using a number of gestures that an older person can memorize. They will probably be even less incentivized to do this (learning new things sucks) if they already have a solution (remote), even though they really love you. I'm just warning you.

I recommend you use OpenNI and NITE. Note that the current version of OpenNI (2) does not have Kinect support. You need to use OpenNI 1.5.4 and look for the SensorKinect093 driver. There should be some gesture code that works for that (googling OpenNI Gesture yields a ton of results). If you're using something that expects OpenNI 2, be warned that you may have to write some glue code.

The basic control set would be Volume +/-, Channel +/-, Power on/off. But that will be frustrating if she wants to go from Channel 03 to 50.

I don't know how low-level you want to go, but a really, REALLY simple gesture recognize could look at horizontal and vertical swipes of the right hand exceeding a velocity threshold (averaged). Be warned: detected skeletons can get really wonky when people are sitting (that's actually a bit of what my PhD is on).

escapecharacter
  • 954
  • 2
  • 12
  • 29
  • Hi. Thank you very much for the answer. I've been looking into OpenNI and NITE and they do seem like the preferable options. I've been having trouble finding as you say the drivers for these things, so thanks for specifying the exact versions I need. I've seen some example code for this, but I'm not 100% sure on how I actually build this. If I use say Visual C++, I'm not greatly experienced with incorporating all the required libraries into the project for compiling. Would you be able to shine a little light on how I should go about this? I appreciate your time in answering this question. – user3046180 Nov 28 '13 at 23:14
  • You say you want to use Linux, yet you're compiling on Visual C++? I'm mostly familiar with Visual C++ and OSX, unfortunately. I'm not as experienced with non-IDE compilers. What I would recommend is finding a sample project for your preferred operating system that uses OpenNI (preferably 1.x version) to get the skeleton. Poke around to find a reference to the user's right hand and track that. – escapecharacter Nov 29 '13 at 04:44
  • I think I'll start by getting this to work on Windows as you say with the appropriate versions so I can have the functionality of the IDE help me out with compiling, and move to Linux another time. But yes, you have been extremely helpful, thanks! – user3046180 Nov 29 '13 at 14:45
  • I've found here a fantastic guide to coding the Kinect for gesture recognition without a GUI or anything to do with windows! http://www.primesense.com/wp-content/uploads/2013/04/PrimeSense_NiTE2API_ProgTutorialGuide_C++Samples_docver0.2.pdf – user3046180 Nov 29 '13 at 18:09
  • In fact, scratch that. It's for OpenNI2 and Nite2 – user3046180 Nov 29 '13 at 18:11