20

I'm building a boardgame in WebGL. The board can be rotated/zoomed. I need a way to translate a click on the canvas element (x,y) into the relevant point in 3D space (x, y, z). The ultimate result is that I want to know the (x, y, z) coordinate that contains the point that touches the object closest to the user. For instance, the user clicks a piece, and you imagine a ray traveling through 3D space that goes through both the piece and the game board, but I want the (x, y, z) coord of the piece at the point where it was touched.

I feel like this must be a very common problem, but I can't seem to find a solution in my googles. There must be some way to project the current view of the 3D space into 2D so you can map each point in 2D space to the relevant point in 3D space. I want to the user to be able to mouse over a space on the board, and have the spot change color.

seibelj
  • 890
  • 2
  • 10
  • 22
  • You're not talking about this are you? http://www.3dkingdoms.com/selection.html#ray Just drop a normal from the camera lens to the plane and figure out the POI. – Incognito Sep 09 '11 at 19:54
  • I think [this](http://stackoverflow.com/questions/2532693/converting-2d-mouse-coordinates-to-3d-space-in-opengl-es/2532701#2532701) is the general description of what I want to do but I can't figure out an algorithm from his description. – seibelj Sep 09 '11 at 20:21
  • There must be some sort of calculation from imagining the viewport as a flat plane, located at (0,0,0) and the center of the rectangle at the origin. I will ponder this more, I'm confident there is an elegant solution. – seibelj Sep 09 '11 at 20:39

3 Answers3

30

You're looking for an unproject function, which converts screen coordinates into a ray cast from the camera position into the 3D world. You must then perform ray/triangle intersection tests to find the closest triangle to the camera which also intersects the ray.

I have an example of unprojecting available at jax/camera.js#L568 -- but you'll still need to implement ray/triangle intersection. I have an implementation of that at jax/triangle.js#L113.

There is a simpler and (usually) faster alternative, however, called 'picking'. Use this if you want to select an entire object (for instance, a chess piece), and if you don't care about where the mouse actually clicked. The WebGL way to do this is to render the entire scene in various shades of blue (the blue is a key, while red and green are used for unique IDs of the objects in the scene) to a texture, then read back a pixel from that texture. Decoding the RGB into the object's ID will give you the object that was clicked. Again, I've implemented this and it's available at jax/world.js#L82. (See also lines 146, 162, 175.)

Both approaches have pros and cons (discussed here and in some of the comments after) and you'll need to figure out which approach best serves your needs. Picking is slower with huge scenes, but unprojecting in pure JS is extremely slow (since JS itself isn't all that fast) so my best recommendation would be to experiment with both.

FYI, you could also look at the GLU project and unproject code, which I based my code loosely upon: http://www.opengl.org/wiki/GluProject_and_gluUnProject_code

sinisterchipmunk
  • 1,993
  • 17
  • 21
  • To clarify: picking in WebGL is usually faster than unprojecting purely because JS is slow, and picking takes advantage of the GPU for hardware acceleration, thus bypassing the JS bottleneck. In a traditional C implementation, unprojecting is probably the way to go, especially if you only need to check a very small number of triangles. – sinisterchipmunk Sep 10 '11 at 18:30
  • Wow, thanks so much, I'll look into all of this. This perfectly answered my question. – seibelj Sep 13 '11 at 14:40
  • So if I need to know exact triangle/vertice, picking is not sufficient? – Drew Sep 20 '11 at 03:01
  • Correct. You could hack your way around this by drawing each triangle with a different picking color, but this would be horribly slow (I doubt it would be viable in JavaScript, but stranger things have happened) since it would require a new pass for each triangle. For exact triangle/vertex information, your best bet is unprojection. – sinisterchipmunk Sep 20 '11 at 11:42
  • Just for clarification, the term "picking" is not related to any specific technique used. Also depending on the structure of the code picking via colorids can become really slow when having a few thousand objects visible on the screen. That's why i would recommend ray picking with a decent spatial subdivision applied(to both the scene and the mesh triangles). – LJᛃ Nov 28 '13 at 00:10
2

I'm working on this problem at the moment - the approach I'm taking is

  1. Render objects to pick buffer each with unique colour
  2. Read buffer pixel, map back to picked object
  3. Render picked object to buffer with each pixel colour a function of Z-depth
  4. Read buffer pixel, map back to Z-depth
  5. We have picked object and approximate Z for the pick coords
xeolabs
  • 1,369
  • 1
  • 9
  • 16
-1

culted from one of the threads. not sure about (x,y,z) but you can get the canvas(x,y) using

getBoundingClientRect()

function getCanvasCoord(){
  var mx = event.clientX;
  var my = event.clientY;
  var canvas = document.getElementById('canvasId');
  var rect = canvas.getBoundingClientRect();// check if your browser supports this
  mx = mx - rect.left;
  my = my - rect.top;
  return {x: mx , y: my};
}
Amit
  • 15,217
  • 8
  • 46
  • 68
Naruto
  • 37
  • 1
  • 6