Detecting touch position on 3D objects in openGL

Question

I have created a 3D object in opengl for one of my application. The object is something like a human body and can be rotated on touch. How can I detect the position of touch on this 3D object. Means if the user touches the head, I have to detect that it is the head. If touch is on the hand, then that has to be identified. It should work even if the object is rotated to some other direction. I think the coordinates of touch on the 3D object is required.
This is the method where I am getting the position of touch on the view.

- (void) touchesBegan: (NSSet*) touches withEvent: (UIEvent*) event
{
    UITouch* touch = [touches anyObject];
    CGPoint location  = [touch locationInView: self];
    m_applicationEngine->OnFingerDown(ivec2(location.x, location.y));
}

Can anyone help? Thanks in advance!

score 5 · Accepted Answer · answered Aug 20 '13 at 06:03

5

Forget about RayTracing and other Top Notch Algorithms. We have used a simple trick for one of our applications(Iyan 3D) on App Store. But this technique need one extra render pass everytime you finish rotating the scene to a new angle. Render different objects (head, hand, leg etc) in different colors (not actual colors but unique ones). Read the color in the rendered image corresponding to the screen position. You can find the object based on its color.

In this method you can use change rendered image resolution to balance accuracy and performance.

answered Aug 20 '13 at 06:03

codetiger

2,650
20
37

+1 - It's very simple and easy to implement technique. Here you have more information and source code in Processing(something like Java) - http://wiki.processing.org/w/Picking_with_a_color_buffer – cyriel Aug 20 '13 at 09:17
Search for Render to texture for rendering. Then it should be easy for you to find colors. http://stackoverflow.com/questions/8439697/opengl-es-2-0-render-to-texture – codetiger Aug 20 '13 at 12:45
This is the correct answer. Make sure you turn off lighting effects, such as ambient/specular lighting – Jason Aug 20 '13 at 13:06
You just have to calculate Vertex position in the vertex shader and feed the corresponding color value in the fragment shader. The RTT is simple and easy to implement. You can increase the accuracy by using 2048x2048 but thats too much to detect touch. We used 1024x512 and it works very well. – codetiger Aug 20 '13 at 17:11
my object have a single color as of requirement.can not render it in different colors. – iOS Developer Aug 27 '13 at 06:19
To render for touch detection, you can use unique colors, which will not be displayed to user. And you will be rendering this only to texture. for touch calculations alone, not to FrameBuffer. – codetiger Aug 27 '13 at 07:42
@codetiger ok..will try this..as i m new to this,have to research before implementing.thanks for your info. – iOS Developer Aug 28 '13 at 06:30
These types of questions and answers are broad to the end and make absolutely no sense without any context behind them. Where is the code? How did you get to this conclusion? Stackoverflow closes questions with exemplary details and diagnosis... Are people real in here? – IOviSpot Nov 03 '21 at 15:24

illeyezur · Answer 2 · 2013-08-21T08:29:37.367

To determine the 3D location of the object I would suggest ray tracing.

Assuming the model is in worldspace coordinates you'll also need to know the worldspace coordinates of the eye location and the worldspace coordinates of the image plane. Using those two points you can calculate a ray which you will use to intersect with the model, which I assume consists of triangles.

Then you can use the ray triangle test to determine the 3D location of the touch, by finding the triangle that has the closest intersection to the image plane. If you want which triangle is touched you will also want to save that information when you do the intersection tests.

This page gives an example of how to do ray triangle intersection tests: http://www.scratchapixel.com/lessons/3d-basic-lessons/lesson-9-ray-triangle-intersection/ray-triangle-intersection-geometric-solution/

Edit:

Updated to have some sample code. Its some slightly modified code I took from a C++ raytracing project I did a while ago so you'll need to modify it a bit to get it working for iOS. Also the code in its current form wouldn't even be useful since it doesn't return the actual intersection point but rather if the ray intersects the triangle or not.

// d is the direction the ray is heading in
// o is the origin of the ray
// verts is the 3 vertices of the triangle
// faceNorm is the normal of the triangle surface
bool
Triangle::intersect(Vector3 d, Vector3 o, Vector3* verts, Vector3 faceNorm)
{
    // Check for line parallel to plane
    float r_dot_n = (dot(d, faceNorm));

    // If r_dot_n == 0, then the line and plane are parallel, but we need to 
    // do the range check due to floating point precision
    if (r_dot_n > -0.001f && r_dot_n < 0.001f)
        return false;

    // Then we calculate the distance of the ray origin to the triangle plane
    float t = ( dot(faceNorm, (verts[0] - o)) / r_dot_n);
    if (t < 0.0)
        return false;

    // We can now calculate the barycentric coords of the intersection
    Vector3 ba_ca = cross(verts[1]-verts[0], verts[2]-verts[0]);
    float denom = dot(-d,  ba_ca);

    dist_out = dot(o-verts[0], ba_ca) / denom;
    float b = dot(-d, cross(r.o-verts[0], verts[2]-verts[0])) / denom;
    float c = dot(-d, cross(verts[1]-verts[0], o-verts[0])) / denom;

    // Check if in tri or if b & c have NaN values
    if (  b < 0 || c < 0 || b+c > 1 || b != b || c != c)
        return false;

    // Use barycentric coordinates to calculate the intersection point
    Vector3 P = (1.f-b-c)*verts[0] + b*verts[1] + c*verts[2];

    return true;
}

The actual intersection point you would be interested in is P.

Can i get any sample codes, using this ray triangle intersection in iOS? — iOS Developer, Aug 16 '13 at 05:46
I added some sample code which lays out what calculations need to be done. Its in C++ because I took it from one of my old projects, but it should be easily understandable. Also the return value of the function isn't what you are looking for, but the end of the function the intersection point is calculated so you should be able to convert it to a useful format for youself. — illeyezur, Aug 21 '13 at 08:32

score 2 · Answer 3 · answered Aug 17 '13 at 00:19

2

Ray tracing is an option and is used in many applications for doing just that (picking). The problem with ray tracing is that this solution is a lot of work to get a pretty simple basic feature working. Also ray tracing can be slow but if you have only one ray to trace (the location of your finger say), then it should be okay.

OpenGL's API also provides a technique to pick object. I suggest you look at for instance: http://www.lighthouse3d.com/opengl/picking/

Finally a last option would consist of projecting the vertices of an object in screen space and use simple 2d techniques to find out which faces of the object your finger overlaps.

answered Aug 17 '13 at 00:19

user18490

3,546
4
33
52

ES doesn't provide the picking API but he could do a similar thing by performing a render with each polygon rendered a unique colour and then using `glReadPixels` to grab the colour at the touch location, or even by coming up with a slightly more elaborate but probably more efficient system based on painting a very close surface with a hole and continuing on with divided geometry and the occlusion query (which is provided on iOS as of iOS 5 if memory serves). If segments are closed then you could do the occlusion as you draw by putting the cover in the background and inverting the depth test. – Tommy Aug 17 '13 at 00:58

Detecting touch position on 3D objects in openGL

3 Answers3