18

There are Google results and stackoverflow posts that appear to answer this question, but the simple fact is I can't understand them. No matter how much I read, I can't get my head around quaternions and Euler angles and Rodriguez transforms and all that.

Anyone up for explaining to me like I'm 12 how to get from the rotation and translation vectors returned by OpenCV's solvePnP() method to xyz Position and xyz Rotation values that I can plug into a 3d graphics application?

This is for an augmented reality application in c++. I have the OpenCV part up and running, tracking a markerboard and sending rotation and translation vectors to the graphics program, but I have no idea how to use that information to pose my 3d object.

I really do want to understand the math behind this, and will appreciate any patient explanation of the theory here. But I'll also settle for a pointer to some code I can copy/paste and learn the theory another day. In fact I seem to learn stuff like this faster by seeing concrete code and working backward to the theory.

EDIT: Like there's this... which clearly should point me in the right direction, but it may as well be the plans for a time machine for all I know. It's occurred to me that I may be asking for remedial high school math, but it can't be the first time.

EDIT: Here's an example of the rotation vector and translation vector being returned from solvePnP()... converted into XML for the trip to the graphics application. Note that every one of these I've seen contains three rows and one column.

<Tvec type_id="opencv-matrix">
  <rows>3</rows>
  <cols>1</cols>
  <dt>f</dt>
  <data>
    -2.50094433e+01 -6.59909010e+00 1.07882790e+02
  </data>
</Tvec>
<Rvec type_id="opencv-matrix">
  <rows>3</rows>
  <cols>1</cols>
  <dt>f</dt>
  <data>
    -1.92100227e+00 -2.11300254e-01 2.80715879e-02
  </data>
</Rvec>
John
  • 565
  • 1
  • 5
  • 14
  • This question should be moved to http://math.stackexchange.com/ – karlphillip Dec 11 '12 at 16:49
  • 13
    See here's the thing with that, karlphillip, if I ask that question in math.stackexchange, I'm gonna get responses replete with things I don't understand. My only hope of understanding this is to see how to do it in code. Then I can understand the math bit. It's just how the brain I was issued works. – John Dec 11 '12 at 17:12

3 Answers3

19

Initially I knew nothing about what formats OpenCV uses. So some research first. Looking at the documentation for solvePnP, it appears like its returning a matrix. (The Rodrigues2 function mentioned in an older version in the docs appears to be called Rodrigues these days.)

The description there indicates that you can expect a matrix with either a single row or a single column. The direction of that vector indicates the axis of rotation, while the length (or “norm”) of the vector gives the angle.

So your input is in angle-axis representation. Which you can interpret as a simultaneous rotation about the x, y and z axes. So if by “XYZ rotation” you mean simultaneous rotation, then you are already there. If, however, you mean some combination which rotates first about the x axis and then about the y axis and then about the z axis, you are in a strange setup, and probably should make sure you are using the correct API.

If it is Euler angles you are after, then calling them “XYZ rotation” is somewhat misleading. The Wikipedia article Rotation formalisms in three dimensions gives a list of the various formats, and how to convert between them. In particular, it does give formulas to convert from Axis and angle to matrix and from there to euler angles.

Make sure you are using the correct representation to match your graphics output machinery. Or if you have a choice, try to work directly on the rotation marix representation, as that should be easiest to understand and work with.

MvG
  • 57,380
  • 22
  • 148
  • 276
  • This is great... and thanks for the research you did. I'm definitely after simultaneous rotation, though I never knew it was called that... but I don't see how I'm "already there." I'll post the data being returned from the solvePnP() method as an edit to my original question to demonstrate that I haven't yet made it to XYZ rotation and XYZ position. – John Dec 11 '12 at 16:56
  • @John, I see three numbers, and no reason at all why they should *not* describe simultaneous rotations. All numbers are in the range [-π, π] and so is the norm of the vector. WHere exactly do you detect an error? What is your graphics application? – MvG Dec 11 '12 at 21:02
  • @MvG... OK, that's interesting. I'd thought those numbers often went well outside the range of radians, but looking more closely at the data getting spat out I see the numbers above pi or below negative pi have negative exponents. Maybe I was making this all way more difficult than it really is. Still, your explanation of simultaneous rotation was really helpful. I'm closer to not being horrified by this stuff :-) – John Dec 11 '12 at 23:58
  • The documentation link mentioned at first does not lead me anywhere anymore. And I cannot find anything like a documentation anywhere on the web. The closest I find is [this link](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_calib3d/py_pose/py_pose.html#pose-estimation) – Divij Sehgal Jul 01 '18 at 09:11
  • @DivijSehgal: I updated the links to OpenCV documentation. – MvG Jul 03 '18 at 07:32
2

  1. Extend your 3D vectors to homogeneous coordinates (4D). That means a 4D vector with components (X, Y, Z, 1)

  2. Create a 4x4 transformation matrix based on your rotation and translation parameters. It is called an affine transformation. You are on the right track with the article about rotation matrices.

  3. Multiply your coordinate vectors with the transformation matrix.

Check out this wikipedia section: http://en.wikipedia.org/wiki/Transformation_matrix#Affine_transformations

It has an example posed in 2d coordinates, I'm sure you get enough of the idea to extend to 3d.

ypnos
  • 50,202
  • 14
  • 95
  • 141
  • Thanks very much for this... you're making my brain hurt but I think I may be (just) able to figure out some of what you said :) – John Dec 11 '12 at 17:00
2

My guess is that OpenCv already returns XYZ rotations. Take your left hand (left hand rule) and close your fist. Stick your thumb up (z axis), extend your index finger straight in front of you (x axis), then extend your middle finger to the right (y axis).

Any particular orientation vector can be given by up to three rotations. It is important to realize that you can get equivalent rotations. For example, rotate around the z-axis so that your index points to your right. An equivalent rotation would be Y,X, then Y again, without ever involving the Z axis.

However when you use the full rotation matrix (3x3), you don't have to bother with those conventions, because they are all merged together in the matrix. This is what the paper you linked is about, and not time travel, which is fortunate since most of math majors would be able to time travel, thus dooming mankind.

Rotation vectors however usually refers to around which vector in space your object must revolve.

Eric
  • 19,525
  • 19
  • 84
  • 147
  • Your assignment of directions to fingers is—uncommon. Usually you have thumb=x, index=y and middle=z. As depicted in the article you referenced. But that doesn't affect the essence of your post. – MvG Dec 11 '12 at 21:21
  • I know... I come from an aerospace background, Z is always the elevation to the sky but my friend from video game disagrees. – Eric Dec 12 '12 at 00:35