0

I have an idea for an app that takes a printed page with four squares in each corner and allows you to measure objects on the paper given at least two squares are visible. I want to be able to have a user take a picture from less than perfect angles and still have the objects be measured accurately.

I'm unable to figure out exactly how to find information on this subject due to my lack of knowledge in the area. I've been able to find examples of opencv code that does some interesting transforms and the like but I've yet to figure out what I'm asking in simpler terms.

Does anyone know of papers or mathematical concepts I can lookup to get further into this project?

I'm not quite sure how or who to ask other than people on this forum, sorry for the somewhat vague question.

vividpk21
  • 384
  • 1
  • 11
  • This would work for 2D objects flat on the paper. As soon as objects stick out over the paper, or have volume, your idea will no longer work. – Cris Luengo Dec 30 '18 at 17:39
  • Why's that? I want to turn the original (non-perfect) angled image into an image with perpendicular angle to the paper then use pixel widths to determine widths and such, I dont need to be able to measure in 3d at any given point in time. – vividpk21 Dec 31 '18 at 00:49
  • 1
    Things closer to the camera are larger than things further away. Without 3D vision (using stereo cameras or a range camera), you don't know how far away things are, and therefore cannot correct for this. Your method will calibrate sizes across the 2D plane defined by the sheet of paper. Something that is closer to the camera would be projected onto that 2D plane, and consequently be measured to be larger than it actually is. – Cris Luengo Dec 31 '18 at 07:22
  • see related QAs: [Transformation of 3D objects related to vanishing points and horizon line](https://stackoverflow.com/a/53303948/2521214) and [Birdseye view without loss of data](https://stackoverflow.com/a/39316776/2521214) and [Which is the best way to estimate measure of photographed things?](https://stackoverflow.com/a/34085449/2521214) and [De-skew characters in binary image](https://stackoverflow.com/a/30273878/2521214) without images its hard to narrow them down. However I am voting to Close as you are asking for Off site tutorial/help/docs instead of a specific programming question – Spektre Dec 31 '18 at 09:53
  • @cris You have the reference shapes of known size, meaning you should be able to measure the skew/distortion of the picture or no? An example would be having your 1x1 square and having the picture take such that it's rotated away from you in the z but none on the x or y. Meaning objects at the base of the picture would seem larger as they are closer. But you should be able to grab the change I'm pixel size of the square at its tallest point and lowest point giving you a way to check rotation. – vividpk21 Dec 31 '18 at 18:06

2 Answers2

2

What you describe is very reminiscent of augmented reality marker tracking. Maybe you can start by searching these words on a search engine of your choice.

A single marker, if done correctly, can be used to identify it without confusing it with other markers AND to determine how the surface is placed in 3D space in front of the camera.

But that's all very difficult and advanced stuff, I'd greatly advise to NOT try and implement something like this, it would take years of research... The only way you have is to use a ready-made open source library that outputs the data you need for your app.

It may even not exist. In that case you'll have to buy one. Given the niché of your problem that would be perfectly plausible.

pid
  • 11,472
  • 6
  • 34
  • 63
1

Here I give you only the programming aspect and if you want you can find out about the mathematical aspect from those examples. Most of the functions you need can be done using OpenCV. Here are some examples in python:

  1. To detect the printed paper, you can use cv2.findContours function. The most outer contour is possibly the paper, but you need to test on actual images. https://docs.opencv.org/3.1.0/d4/d73/tutorial_py_contours_begin.html

  2. In case of sloping (not in perfect angle), you can find the angle by cv2.minAreaRect which return the angle of the contour you found above. https://docs.opencv.org/3.1.0/dd/d49/tutorial_py_contour_features.html (part 7b).

  3. If you want to rotate the paper, use cv2.warpAffine. https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_geometric_transformations/py_geometric_transformations.html

  4. To detect the object in the paper, there are some methods. The easiest way is using the contours above. If the objects are in certain colors, you can detect it by using color filter. https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_colorspaces/py_colorspaces.html

Ha Bom
  • 2,787
  • 3
  • 15
  • 29