4

I am trying to run a simple photo stitching example using OpenCV XCFramework in an iOS app, but it fails with the following error during the call to stitcher->stitch(imgs, pano)

ibc++abi: terminating with uncaught exception of type cv::Exception: OpenCV(4.6.0-dev) /Users/gpdawson/Documents/Apps/OpenCV/opencv/modules/imgproc/src/resize.cpp:4065: error: (-215:Assertion failed) inv_scale_x > 0 in function 'resize'

The code that is being called is as follows:

+ (void)testStitchSimple {

UIImage *uiImage;
cv::Mat opencvImage1;
cv::Mat opencvImage2;
cv::Mat opencvImage3;
vector<cv::Mat> imgs;

uiImage = [UIImage imageNamed:@"first.jpg"];
UIImageToMat(uiImage, opencvImage1);
imgs.push_back(opencvImage1);

uiImage = [UIImage imageNamed:@"second.jpg"];
UIImageToMat(uiImage, opencvImage2);
imgs.push_back(opencvImage2);

uiImage = [UIImage imageNamed:@"third.jpg"];
UIImageToMat(uiImage, opencvImage3);
imgs.push_back(opencvImage3);

Stitcher::Mode mode = Stitcher::PANORAMA;
Stitcher *stitcher = Stitcher::create(mode);  // THIS IS THE CAUSE - SEE ANSWER

Mat pano;
Stitcher::Status status;
status = stitcher->stitch(imgs, pano); // FAILS WITH RESIZE ERROR MESSAGE

}

Note that I have tried using different source images and also reading the images from file directly into opencvImage using OpenCV's imread() function. (The image types are CV_8UC4 according to OpenCV's typeToString function).

I've also tried calling estimateTransform instead of stitch and error is the same. This is presumably because stitch itself calls estimateTransform, which is where the same error must be occuring in both cases.

Any help with finding the cause of this error or else at least working around it would be much appreciated.

gpdawson
  • 713
  • 6
  • 15
  • (1/2) The error seems to originate by an internal method called by the stitcher. The method is `cv::resize`. Now, tracking where is the function call in the source code, it seems that originates from another internal method called [`Stitcher::composePanorama`](https://github.com/opencv/opencv/blob/4.x/modules/stitching/src/stitcher.cpp#L129).... – stateMachine Dec 22 '22 at 03:53
  • (2/2)... Tracking what could potentially go wrong here is a little tricky, as `resize` is called multiple times within the method. Something is up with the size of the images you are passing to the stitcher. Can you try playing with the sizes (downsampling them) before giving them to the stitcher? Are all the images of the exact same size? – stateMachine Dec 22 '22 at 03:55
  • The images are all the same size. I am using the exact images as provided in this python stitching example: https://github.com/niconielsen32/ComputerVision/tree/master/imageStitching – gpdawson Dec 22 '22 at 04:30
  • I did also try using my own images - 3 jpeg photos of same size taken on iPhone - but same error occurs – gpdawson Dec 22 '22 at 04:32
  • Did you try to set confidence threshold? For example `stitcher->setPanoConfidenceThresh(0.0);` right after create – Liastre Dec 22 '22 at 08:49
  • @Liastre I did try changing a range of stitcher settings without any change to the result. And I wouldn't really expect it to anyway - this same stitching example (without changing any defaults) seems to work fine when run using Python. See https://www.youtube.com/watch?v=Zs51cg4mb0k – gpdawson Dec 22 '22 at 22:08
  • @gpdawson I've noticed you are using 4.6.0-dev build, try to use stable release, preferably prebuilt one, just to be sure it's not library bug – Liastre Dec 23 '22 at 15:38
  • @Liastre I also tried v4.6.0 (not dev) and v3.4.16 - both fail with same error in call to function resize. – gpdawson Dec 23 '22 at 20:34
  • Have you tried this exact example running on a PC in pure C++? The images are being read correctly, right? (not empty Mats) Something that seems "curious" is the fact that you are loading JPG images, but the image types seem to be CV_8UC4, which indicates the presence of a 4th channel, commonly this should be the alpha channel, but JPGs do not have alpha channel... – stateMachine Dec 24 '22 at 01:43
  • @stateMachine I don't have access to running it in pure C++. But I have now double-checked reading the images in via imread() function. When I do this the image format is CV_8UC3 (so I guess no alpha channel now). But sadly the final result is still exactly the same ie. the same exception when stitch is called. – gpdawson Dec 24 '22 at 02:31
  • I'm out of suggestions. I'd double check the size of each image inside the vector before passing it to the stitcher, just to be sure all images have the same size and are `>0 `. Lastly, you seem to create the mat object `Mat pano` but there's no `cv` namespace i.e, `cv::Mat pano`. Same with `std::vector `. – stateMachine Dec 24 '22 at 03:44
  • @stateMachine Thanks for your suggestions - appreciated! Yes I am able to preview the vector contents and all images are the same size in the array. I did also see the cv namespace issue you saw but not the std:: one. However, sadly, these have also made no difference. :-( – gpdawson Dec 24 '22 at 04:18

1 Answers1

1

I have discovered the cause of the problem. My code to create the stitcher object is apparently faulty, and is presumably a result of inappropriately mixing Objective-C object pointer with C++ Ptr.

Stitcher *stitcher = Stitcher::create(mode); // INCORRECT

cv::Ptr<Stitcher> stitcher = Stitcher::create(mode); // CORRECT

By making the incorrect call, a usable stitcher object was created, but its internal storage (and hence all its initial default settings) were all nil/zero values. Due to these incorrect default values, the subsequent call to stitch was failing.

Once the object was correctly created, the stitching executed successfully.

gpdawson
  • 713
  • 6
  • 15