Even if there is a similar question (unanswered). I will give more details to try to solve it, also this question will be based on C++.
So, imagine we have 3 images and we want to stitch them, but we know that they don't fully overlap, so, to improve the results we should be able to pass a mask on each pair of images to tell the future detector where to look for key points. The documentation is a bit confusing:
Status cv::Stitcher::stitch ( InputArrayOfArrays images,
InputArrayOfArrays masks,
OutputArray pano
)
These functions try to stitch the given images.
Parameters
images: Input images.
masks: Masks for each input image specifying where to look for keypoints (optional).
pano: Final pano.
Returns Status code.
I guess that masks should be a std::vector of std::vectors of cv::Mats of size NxN where N is the amount of images. And each cv::Mat tells the function where to mask the image when matching 2 images. i.e. the cv::Mat in (0,1) will mask image 1 when finding points to match on image 0 and the one in (1,2) will mask the image when finding key points to match between image 1 and 2.
I tried this by declaring masks full of ones:
cv::Mat image1 = cv::imread("my_path_1", cv::IMREAD_UNCHANGED);
cv::Mat image2 = cv::imread("my_path_2", cv::IMREAD_UNCHANGED);
cv::Mat image3 = cv::imread("my_path_3", cv::IMREAD_UNCHANGED);
// The 3 of them have the same size !!
std::vector<cv::Mat> images;
images.push_back(image1);
images.push_back(image2);
images.push_back(image3);
cv::Mat result;
// Deinfe here the masks !!
cv::Mat onesTemplate = cv::Mat::ones(image1.size(), CV_8UC1);
std::vector<std::vector<cv::Mat>> matArray(3, std::vector<cv::Mat>(3, onesTemplate));
auto mode = cv::Stitcher::PANORAMA;
auto stitcher = cv::Stitcher::create(mode);
// Set params here if needed
cv::Stitcher::Status status = stitcher->stitch(images, matArray, result);
if (status == cv::Stitcher::OK) {
std::string fileName = "something"
cv::imwrite(fileName, result);
} else {
std::cerr << "Stitching failed: ";
if (status == cv::Stitcher::ERR_NEED_MORE_IMGS) {
std::cerr << "Insufficient images to perform stitching." << std::endl;
} else if (status == cv::Stitcher::ERR_HOMOGRAPHY_EST_FAIL) {
std::cerr << "Homography estimation failed." << std::endl;
} else if (status == cv::Stitcher::ERR_CAMERA_PARAMS_ADJUST_FAIL) {
std::cerr << "Camera parameter adjustment failed." << std::endl;
} else {
std::cerr << "Unknown error." << std::endl;
}
return 1;
}
When I compile/run this I get the following error:
In file included from /cache/venv/include/opencv4/opencv2/core.hpp:55,
from /cache/venv/include/opencv4/opencv2/features2d.hpp:47,
from /source/stiching_app/src/main.cpp:9:
/cache/venv/include/opencv4/opencv2/core/traits.hpp: In instantiation of ‘struct cv::traits::Type<cv::Mat>’:
/cache/venv/include/opencv4/opencv2/core/mat.inl.hpp:126:60: required from ‘cv::_InputArray::_InputArray(const std::vector<std::vector<_Tp> >&) [with _Tp = cv::Mat]’
/source/stiching_app/src/main.cpp:130:51: required from here
/cache/venv/include/opencv4/opencv2/core/traits.hpp:386:31: error: ‘type’ is not a member of ‘cv::DataType<cv::Mat>’
386 | { enum { value = DataType<T>::type }; };
| ^~~~
gmake[2]: *** [CMakeFiles/stitching.dir/build.make:76: CMakeFiles/stitching.dir/src/main.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/stitching.dir/all] Error 2
Which makes me think I'm just using the wrong object type as cv::DataType< _Tp > only accepts:
A primitive OpenCV data type is one of unsigned char, bool, signed char, unsigned short, signed short, int, float, double, or a tuple of values of one of these types, where all the values in the tuple have the same type.
and not a cv::Mat
The question is, what should I send instead?
Note that I achieve stitching images with my code if I call stitch() with only 2 parameters:
cv::Stitcher::Status status = stitcher->stitch(images, result); // this is ok
