0

Hi i am using an asus xtion pro live camera for my object detection, i am also new to opencv. Im trying to get distance of object from the camera. The Object detected is in 2d image. Im not sure on what should i use to get the information then following up with the calculations to get distance between camera and object detected. Could someone advise me please?

user3868279
  • 7
  • 1
  • 8

3 Answers3

0

In short: You can't.

You're losing the depth information and any visible pixel in your camera image essentially transforms into a ray originating from your camera.

So once you've got an object at pixel X, all you know is that the object somewhere intersects the vector cast based on this pixel and the camera's intrinsic/extrinsic parameters.

You'll essentially need more information. One of the following should suffice:

  • Know at least one coordinate of the 3D point (e.g. everything detected is on the ground or in some known plane).
  • Know the relation between two projected points:
    • Either the same point from different positions (known camera movement/offset)
    • or two points with significant distance between them (like the two ends of some staff or bar).

Once you've got either, you're able to use simple trigonometry (rule of three) to calculate the missing values.

Since I initially missed this being a camera with an OpenNI compatible depth sensor, it's possible to build OpenCV with support for that by definining the preprocessor define WITH_OPENNI when building the library.

Mario
  • 35,726
  • 5
  • 62
  • 78
  • Oh okay. So if i were to detect on a known plane such as tabletop with fixed dimensions i would be able to find out the distance and do something like that? https://www.youtube.com/watch?v=35rbeE1c-D4 – user3868279 Nov 02 '14 at 09:00
  • If you know the table and its position/orientation, then you can use that information to reconstruct the 3D coordinates of anything you "see" being in the same plane (i.e. you intersect the mentioned vector originating from the camera with the known plane). This is essentially how augmented reality games with markers (e.g. on the 3DS) work. The known objects/dimensions (markers) are used to determine the position/orientation of something else (table or ground in this case). – Mario Nov 02 '14 at 11:21
  • Hi could you direct me to tutorials on how to find coordinates of a known plane? – user3868279 Nov 06 '14 at 14:11
  • You'll have to define them yourself. For example, set one corner of the table to be (0, 0). The other one can be (width, height). Once those are done, it's simple math calculating the interception between your vectors originating from the camera and that plane. – Mario Nov 06 '14 at 20:41
  • Erm.. sorry for bothering you but i still don't quite understand.. Is there any coding reference that i can refer to? I'm having quite a hard time figuring out. – user3868279 Nov 09 '14 at 06:59
  • @user3868279 Have a look at [this tutorial explaining 3D reconstruction](http://docs.opencv.org/trunk/doc/py_tutorials/py_calib3d/py_depthmap/py_depthmap.html#py-depthmap). It's meant for Python, yet you should be able to apply it. Also have you read the answer from user1500287? I didn't notice that this is indeed a special camera with a depth sensor, so you could indeed read the depth from the camera (but not using OpenCV). – Mario Nov 09 '14 at 08:26
  • You mean opencv won't be able to get the readings fromd depth map?? But from what i found, opencv2.4.9 shows that opencv should be able to get depth map by enabling the with_openni in the cmake.config. The problem is that i can't seem to find how to enable it. Also, i've read the tutorial explaining 3d reconstruction but somehow i don't really understand what it meant... To be honest, i'm quite lousy at this so i need like full tutorials or codes or smth that would enable me to understand better. Thank You – user3868279 Nov 20 '14 at 03:42
  • @user3868279 Okay, if that's indeed an option now, then you should be able to do it with OpenCV as well. Haven't used it for quite some time though. Try running CMake in your build directory: `cmake -DWITH_OPENNI .`. This should set the flag and enable building with it. As an alternative, you can just look for that value in your `CMakeCache.txt` and update it there. – Mario Nov 20 '14 at 08:04
  • @user3868279 Which part or comment are you referring to? If it's about the Python tutorial, then you should be able to skip it, since you're able to get that from OpenNI. – Mario Nov 25 '14 at 08:54
  • Ok.. So what you are tell me is that, by using OpenNI, i should be able to get the data of depth map and i should be able to do detecting an object on a known plane or something like that right? Sorry if i'm inferring it wrongly cause i'm really quite confused with this. – user3868279 Nov 26 '14 at 09:10
  • @user3868279 With OpenNI (or through the interface built into OpenCV) you should be able to obtain a depth map, i.e. a grayscale representatiobn of your camera's view. But rather than depicting actualcolors color values, the pixels will represent the range between your camera and an object (e.g. white is close up, black is far away) or something like that. I haven't used OpenNI so far, so can't really talk from experience here. – Mario Nov 26 '14 at 09:15
  • Hi erm recently i've tried using this https://code.google.com/p/find-object/ to do recognition using surf. Problem is, i don't know how to convert the distance to real world coordinates. – user3868279 Dec 17 '14 at 08:23
  • @user3868279 Basic idea is the fact that based on your field of view you're able to create a vector pointing from your camera into the filmed scene. E.g. the pixel in the center of the image would point straight in the camera's direction. Once you've got that vector, you can use the known distance along it (your depth!) to determine the actual point in 3D space. – Mario Dec 17 '14 at 08:46
  • Oh.. i kind of understand what you are saying, but the problem is i don't know how to apply it... – user3868279 Dec 24 '14 at 08:38
  • As u can see, i'm trying to use this coding in the aspect of calculating the distance. Althought i'm not sure, it does not seem like it's using any depth calculation. http://robotica.unileon.es/mediawiki/index.php/Objects_recognition_and_position_calculation_(webcam) – user3868279 Dec 24 '14 at 08:44
  • @user3868279 First calculate the vector of the direction the camera is facing. Then determine how to rotate it based on the given pixel coordinate in the image. Assuming the fov is 90° and your pixel is on the left border of the image, you'd have an angle of -45°. In a similar way, if the pixel is half way to the right border, you'd have an angle of +22.5°. In a similar way you'll get an angle for the vertical axis. Then rotate your camera vector based on this. Last but not least, you'll be able to use the depth of the selected pixel to scale the length of the vector. – Mario Dec 24 '14 at 11:59
  • @user3868279 The result will be a vector originating from the camera and pointing to the detected pixel's representation in the 3D world. Just add it to the coordinates of the camera and you'll get the absolute 3D coordinates of the given point. – Mario Dec 24 '14 at 12:00
  • Hi, right now i'm trying to use this code in http://robotica.unileon.es/mediawiki/index.php/Objects_recognition_and_position_calculation_(webcam) the part where i have to do this :find_package(OpenCV "VERSION" REQUIRED) include_directories(${OpenCV_INCLUDE_DIRS}) ... target_link_libraries("PROGRAM_NAME" ${OpenCV_LIBS}) could you help me understand this part? I tired following the instructions but it ended up with a build error. If possible please instruct me on what to do or which part am i wrong. Thank You very much for your continuous help. – user3868279 Jan 09 '15 at 07:09
  • @user3868279 You should create a new question here. CMake should lookup the directories/paths or you might have to set them yourself. Can't really fit that into a comment, especially not without knowing the actual error message. – Mario Jan 09 '15 at 07:46
  • Sorry. I did create a new question, here is the link http://stackoverflow.com/questions/27853188/build-errors-while-following-the-instructions-given-on-a-website/27866453#27866453 – user3868279 Jan 10 '15 at 02:43
  • @user3868279 No problem, it should help you getting answers faster though (since it's far more visible and essentially off-topic here in the comments). – Mario Jan 10 '15 at 08:43
0

I don't like to be the one breaking this to you but what you are trying to do is either impossible or extremely difficult with a single camera.

You need to have the camera moving, record a video of it and use a complex technique such as this. Usually 3d information is created from at least 2 2d images taken from 2 different places. You also need to know quite precisely the distance and the rotation between the two images. The common technique is to have 2 cameras with a precisely measured distance between the two.

Community
  • 1
  • 1
Eric
  • 19,525
  • 19
  • 84
  • 147
0

The Xtion is not a basic webcam. It's a stereo-scopic depth sensing cam similar to Kinect and Primesense. The main API for this is OpenNI - see http://structure.io/openni.

user1500287
  • 21
  • 1