3

I am writing a Eye On Hand Calibration Program.

To do that, I am moving the camera mounted on the robot arm to 20 different positions, looking at a single aruco marker.

The translation vector is very stable, but the rotation axes flicker, introducing an error into the resulting calibration matrix.

Therefore, I would like to average X number of frames' rotation vectors (The aruco library does return rotation vectors and translation vectors separately).

Here is the important part of the code

    cv::aruco::detectMarkers(image, dictionary, markerCorners, markerIds, parameters, rejectedCandidates);

    outputImage = image.clone();

    cv::aruco::drawDetectedMarkers(outputImage, markerCorners, markerIds);
    cv::aruco::estimatePoseSingleMarkers(markerCorners, 0.05, camMatrix, distCoeffs, rvecs, tvecs);

rvecs is actualy a vector of rotation vectors, with only one member because there is only one aruco marker.

If a marker is found in the frame then,

        if (rvecs.size() == 1) {  // There is one marker good frame
          framesFound++;
          for (int i = 0; i < 3; i++) {
            avgRvecs[i] =+ rvecs[0][i];
            avgTvecs[i] =+ tvecs[0][i];
          }       
        }

And after all the desired frames to average have been processed,

   if (framesFound == 0 ) {  // No frames with markers...
   } else {
    for (int i = 0; i < 3; i++) {
        avgRvecs[i] = avgRvecs[i] / framesFound;
        avgTvecs[i] = avgTvecs[i] / framesFound;
   }       

   cv::drawFrameAxes(outputImage, camMatrix, distCoeffs, avgRvecs, avgTvecs, 0.1);

With a single frame I get

enter image description here

With 10 averaged frames I get

enter image description here

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
Dave B
  • 51
  • 5
  • you could try to average the rotation _matrices_ calculated from rotation vectors (make sure the rotation matrix is still orthogonal...). these rvecs are axis-angle representations. you can't just add/average those. – Christoph Rackwitz Oct 19 '22 at 08:42
  • Thanks Christoph! I'll try that using cv::rodriguez to convert, and report back... – Dave B Oct 19 '22 at 11:53
  • I know this may be too late, but you cannot take the simple average of a set of rotation matrices. They are in the space of SO(3), and addition of two rotation matrices is not in SO(3). One of the easier/simpler options would be to convert the rvecs to Euler angles or quaternions and average them, and convert back yo rvec. – Kani Nov 17 '22 at 19:17

1 Answers1

1

Because the pose estimation of Aruco markers is usually done with IPPE (the algorithm under the hood of solvePnP), you might have some "singularities" in the rotation results.

Averaging the rotations can be a good solution as it works as a low-pass filter, though you need to remember that if you are seeking precision, this might not be the most appropriate filter to use.

Most of the time, we like to manipulate Euler angles, Direct Cosine Matrix, or Rodriguez angles. Unfortunately, none of those is the ideal solution to average angular values. If you want to manipulate angles without mathematical errors, I definitely recommend having a look at quaternions.

Some existing posts suggest interesting approaches with quaternions :

"Average" of multiple quaternions?

https://math.stackexchange.com/questions/1984608/average-of-3d-rotations

Math can be a little scary but the posts come with well written code examples.

AlixL
  • 372
  • 2
  • 13