How can I rewrite this warp-affine using OpenCV?

Question

I'm trying to optimize this code, in particular:

bool interpolate(const Mat &im, float ofsx, float ofsy, float a11, float a12, float a21, float a22, Mat &res)
{         
   bool ret = false;
   // input size (-1 for the safe bilinear interpolation)
   const int width = im.cols-1;
   const int height = im.rows-1;
   // output size
   const int halfWidth  = res.cols >> 1;
   const int halfHeight = res.rows >> 1;
   float *out = res.ptr<float>(0);
   for (int j=-halfHeight; j<=halfHeight; ++j)
   {
      const float rx = ofsx + j * a12;
      const float ry = ofsy + j * a22;
      for(int i=-halfWidth; i<=halfWidth; ++i)
      {
         float wx = rx + i * a11;
         float wy = ry + i * a21;
         const int x = (int) floor(wx);
         const int y = (int) floor(wy);
         if (x >= 0 && y >= 0 && x < width && y < height)
         {
            // compute weights
            wx -= x; wy -= y;
            // bilinear interpolation
            *out++ = 
               (1.0f - wy) * ((1.0f - wx) * im.at<float>(y,x)   + wx * im.at<float>(y,x+1)) +
               (       wy) * ((1.0f - wx) * im.at<float>(y+1,x) + wx * im.at<float>(y+1,x+1));
         } else {
            *out++ = 0;
            ret =  true; // touching boundary of the input            
         }
      }
   }
   return ret;
}

According to Intel Advisor, this is a very time consuming function. In this question I asked how I could optimize this, and someone made me notice that this is warp-affine transformation.

Now, since I'm not the image processing guy, I had to read this article to understand what a warp-affine transformation is.

To my understanding, given a point p=(x,y), you apply a transformation A (a 2x2 matrix) and then translate it by a vector b. So the obtained point after the transformation p' can be expressed as p' = A*p+b. So far so good.

However, I'm a little bit confused on how to apply cv::warpAffine() to this case. First of all, from the function above interpolate() I can see only the 4 A components (a11, a12, a21, a22), while I can't see the 2 b components...Are they ofsx and ofy?

In addition notice that this function returns a bool value, which is not returned by warpAffine (this boolean value is used here at line 126), so I don't know I could this with the OpenCV function.

But most of all I'm so confused by for (int j=-halfHeight; j<=halfHeight; ++j) and for(int i=-halfWidth; i<=halfWidth; ++i) and all the crap that happens inside.

I understand that:

        // bilinear interpolation
        *out++ = 
           (1.0f - wy) * ((1.0f - wx) * im.at<float>(y,x)   + wx * im.at<float>(y,x+1)) +
           (       wy) * ((1.0f - wx) * im.at<float>(y+1,x) + wx * im.at<float>(y+1,x+1));

Is what INTER_LINEAR does, but apart from that I'm totally lost.

So, to test my approach, I tried to do the equivalent of line 131 of this as:

     bool touchesBoundary = interpolate(smoothed, (float)(patchImageSize>>1), (float)(patchImageSize>>1), imageToPatchScale, 0, 0, imageToPatchScale, patch);
     Mat warp_mat( 2, 3, CV_32FC1 );
     float a_11 = imageToPatchScale;
     float a_12 = 0;
     float a_21 = 0;
     float a_22 = imageToPatchScale;
     float ofx = (float)(patchImageSize>>1);
     float ofy = (float)(patchImageSize>>1);
     float ofx_new = ofx - a12*halfHeight - a11*halfWidth;
     float ofy_new = ofy - a22*halfHeight - a21*halfWidth;
     warp_mat.at<float>(0,0) = imageToPatchScale;
     warp_mat.at<float>(0,1) = 0;
     warp_mat.at<float>(0,2) = ofx_new;
     warp_mat.at<float>(1,0) = 0;
     warp_mat.at<float>(1,1) = imageToPatchScale;
     warp_mat.at<float>(1,2) = ofy_new;
     cv::Mat myPatch;
     std::cout<<"Applying warpAffine"<<std::endl;
     warpAffine(smoothed, myPatch, warp_mat, patch.size());
     std::cout<<"WarpAffineApplied patch size="<<patch.size()<<" myPatch size="<<myPatch.size()<<std::endl;
     cv::Mat diff = patch!=myPatch;
     if(cv::countNonZero(diff) != 0){
         throw std::runtime_error("Warp affine doesn't work!");
     }
     else{
         std::cout<<"It's working!"<<std::endl;
     }

And of course at the first time the this is executed, the exception is thrown (so the two methods are not equivalent)...How can I solve this?

Can someone help me please?

As I already written in the comments, the resulting matrix by using the code above is a zero matrix. While this is maPatch obtained by using ofx and ofy instead of ofx_new and ofy_new, while patch has all the values different from zero:

[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 229.78679, 229.5752, 229.11732, 229.09612, 229.84615, 230.28633, 230.35257, 230.70955, 230.99368, 231.00777, 231.20511, 231.63196, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 230.60367, 230.16417, 230.07034, 230.06793, 230.02016, 230.14925, 230.60413, 230.84822, 230.92368, 231.02249, 230.99162, 230.9149, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 232.76547, 231.39716, 231.26674, 231.34512, 230.746, 230.25253, 229.65276, 227.83998, 225.43642, 229.57695, 230.31363, 230.16011, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 234.01663, 232.88118, 232.15475, 231.40129, 223.21553, 208.22626, 205.58975, 214.53882, 220.32681, 228.11552, 229.31509, 228.86545, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 234.04565, 233.00443, 231.9902, 230.14912, 198.0849, 114.86175, 97.901344, 160.0218, 217.38528, 231.07045, 231.13109, 231.10185, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 233.293, 232.69095, 217.03873, 190.56714, 167.61592, 94.968391, 81.302032, 150.72263, 194.79535, 215.15564, 230.01717, 232.37894, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 231.70988, 227.81319, 207.59377, 173.35149, 113.88276, 73.171112, 71.523285, 103.05875, 160.05588, 194.65132, 226.4287, 231.45871, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 231.93924, 224.24269, 199.1693, 150.65695, 103.33984, 79.489555, 77.509094, 87.893059, 122.01918, 168.37506, 219.22086, 231.05161, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 232.2706, 232.12926, 206.97635, 127.69308, 92.714355, 81.512207, 74.89402, 75.968353, 84.518105, 157.07962, 223.18773, 229.92766, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 232.64882, 222.16704, 161.95021, 92.577881, 83.757164, 76.764214, 67.041054, 66.195595, 71.112335, 131.66878, 188.27278, 217.6635, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 234.77202, 231.75511, 178.64326, 104.27015, 95.664223, 82.791382, 67.68969, 72.78054, 72.355469, 104.77696, 172.32361, 204.92691, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 236.49684, 235.5802, 185.34337, 115.96995, 106.85963, 82.980408, 61.703068, 69.540627, 76.200562, 82.429321, 101.46993, 119.75877, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

Example of smoothed:

[229.78679, 229.67955, 229.56825, 229.40576, 229.08748, 228.90848, 229.13086, 229.53154, 229.91875, 230.1864, 230.31964, 230.34709, 230.35471, 230.51445, 230.81174, 230.97459, 231.00513, 231.00487, 231.01001, 231.08649, 231.30977, 231.55736, 231.71651;
 229.71237, 229.63612, 229.65092, 229.72298, 229.65163, 229.58559, 229.68594, 229.8093, 229.91052, 230.0466, 230.22325, 230.43683, 230.67668, 230.87794, 230.98672, 231.02505, 231.03383, 231.03091, 231.02097, 231.03201, 231.09761, 231.17659, 231.23175;
 230.66309, 230.37627, 230.1866, 230.1675, 230.09061, 230.03766, 230.10495, 230.09256, 230.01401, 230.03775, 230.18376, 230.42041, 230.67554, 230.82742, 230.84885, 230.87372, 230.94225, 231.01442, 231.02843, 231.00027, 230.97455, 230.9254, 230.86211;
 232.00514, 231.33768, 230.82791, 230.77686, 230.84599, 230.88741, 230.84238, 230.58279, 230.27737, 230.22282, 230.2531, 230.28053, 230.33743, 230.24406, 229.8969, 229.53674, 229.66661, 230.42201, 230.86761, 230.84827, 230.7677, 230.72296, 230.69333;
 232.84413, 232.07454, 231.4113, 231.24339, 231.31792, 231.42, 231.39203, 231.09439, 230.71797, 230.52229, 230.16359, 229.71872, 229.5307, 228.81219, 226.98767, 224.92525, 225.05101, 228.29745, 230.37059, 230.39821, 230.14323, 230.08817, 230.12051;
 233.69714, 233.27977, 232.63216, 231.97507, 231.61856, 231.50835, 231.37958, 230.94897, 230.22003, 229.17024, 227.78331, 226.92528, 227.3483, 226.49516, 223.07671, 219.54231, 220.02966, 225.84485, 229.56601, 229.69946, 229.2941, 228.91028, 228.47911;
 234.07579, 233.56334, 232.87689, 232.33269, 232.23909, 232.26355, 231.24196, 227.51971, 220.59465, 210.97746, 202.39467, 198.75334, 202.68945, 209.23911, 214.57399, 218.0966, 221.80714, 226.69366, 229.27985, 229.35699, 229.21922, 229.04704, 228.72176;
 234.02943, 233.1526, 232.62421, 232.68416, 232.63794, 232.74126, 230.84375, 220.47586, 197.81956, 164.03839, 136.08931, 125.05849, 134.9079, 158.19888, 186.67014, 209.67909, 223.89606, 229.51706, 230.72685, 230.50046, 230.31461, 230.29973, 230.30855;
 234.04939, 233.55843, 233.05295, 232.52957, 231.76837, 231.33992, 229.65753, 220.00912, 191.89427, 140.79909, 97.534477, 80.921623, 93.553299, 127.26912, 171.24872, 205.13603, 224.29935, 230.74513, 231.68158, 231.38503, 231.22385, 231.26157, 231.31372;
 233.67462, 233.69278, 233.09642, 230.73448, 225.79077, 220.33292, 216.52835, 212.6403, 192.7964, 142.2917, 93.74559, 73.776016, 92.972778, 136.18417, 183.40891, 209.98003, 220.25392, 225.67984, 229.14565, 230.97379, 231.68997, 231.87923, 231.80464;
 233.16579, 232.95818, 232.5157, 227.84683, 212.53104, 193.47, 179.53844, 171.00941, 154.97589, 118.29485, 82.342369, 67.311531, 83.867973, 119.85723, 158.53325, 180.67912, 191.74194, 203.44008, 216.87592, 227.59789, 231.31285, 232.24002, 232.91658;
 232.21611, 231.93192, 231.80423, 227.06053, 208.82571, 183.86725, 160.27481, 136.63663, 112.56454, 89.978371, 73.328209, 66.652176, 73.406273, 90.259987, 113.70027, 138.08961, 159.2791, 178.08627, 201.78604, 223.79007, 230.86775, 231.59146, 232.17819;
 231.5118, 230.38042, 225.97289, 217.07312, 205.34308, 192.29631, 174.19812, 142.59843, 105.71719, 80.45845, 68.488274, 67.021088, 73.29406, 86.493896, 110.19484, 145.04185, 174.52554, 187.26851, 202.64322, 221.51042, 229.94238, 231.48595, 231.08746;
 231.67564, 229.07423, 217.57478, 197.87076, 181.8385, 167.48799, 148.19232, 124.3977, 100.57513, 83.081154, 73.410683, 71.723045, 77.010704, 85.107651, 98.029099, 121.88382, 145.77963, 161.43314, 184.43152, 212.01347, 227.27411, 231.84755, 231.33319;
 232.0773, 231.27109, 227.09813, 218.50165, 206.31781, 182.26494, 144.46196, 115.64604, 99.402679, 87.584351, 79.348366, 76.547188, 79.332504, 82.244148, 86.3069, 100.71764, 122.39668, 147.5081, 179.02258, 210.10269, 226.37909, 231.12947, 230.34335;
 232.11732, 231.67418, 231.89207, 229.20001, 213.83904, 180.2238, 134.82561, 107.20949, 97.260231, 88.765694, 80.533333, 75.941055, 76.372505, 77.851997, 78.464508, 81.875244, 96.896721, 131.28108, 175.47084, 213.05406, 227.81297, 230.31032, 229.60373;
 232.36255, 232.00981, 232.29773, 226.30051, 199.48029, 156.13557, 112.30969, 91.346344, 88.295509, 85.21006, 79.416222, 74.552238, 73.894844, 75.069275, 74.349594, 72.166176, 85.453522, 128.47208, 180.33452, 218.87312, 229.58446, 229.77406, 230.03587;
 232.52425, 231.2455, 226.65468, 210.90804, 174.35748, 128.79022, 92.861343, 79.050415, 78.796555, 76.526512, 71.317635, 67.324234, 67.506172, 69.193619, 68.941025, 67.913399, 82.488945, 124.88449, 171.48178, 203.84958, 215.13747, 221.22523, 228.15715;
 232.74571, 229.80283, 217.69687, 189.34862, 145.52664, 104.71513, 84.893997, 83.699814, 88.473457, 86.446617, 77.834595, 68.74688, 65.925613, 65.426163, 63.241882, 61.236107, 69.682426, 97.213646, 131.60564, 160.99944, 180.75278, 202.22523, 223.85883;
 233.80923, 232.82767, 227.83594, 209.05493, 166.58002, 120.64989, 94.880188, 89.971268, 93.209671, 90.605591, 80.354561, 69.243584, 67.490875, 70.700516, 72.353569, 70.053764, 70.773293, 86.577957, 121.76624, 160.51776, 182.91074, 203.17424, 224.06786;
 235.62155, 235.22169, 234.91901, 223.3783, 181.88362, 132.80327, 104.59508, 97.904762, 98.472153, 91.749123, 79.65731, 69.025223, 66.806007, 70.64135, 75.239159, 74.961838, 73.406227, 83.469612, 118.84832, 161.62743, 181.61127, 192.7933, 203.54196;
 236.851, 236.1096, 235.65253, 224.02559, 182.0352, 134.56085, 111.10134, 106.82736, 105.87054, 95.272148, 80.614365, 68.017456, 61.20583, 62.735069, 69.976379, 72.687195, 71.943336, 75.369637, 89.042145, 106.32064, 116.6455, 127.58019, 139.77493;
 236.09546, 235.84727, 235.44041, 223.06668, 180.65508, 134.57915, 114.13975, 110.49339, 107.15049, 93.355858, 77.559898, 65.277794, 58.067509, 62.642029, 76.700447, 81.800919, 80.054298, 80.085251, 82.980927, 87.177017, 92.031647, 100.26192, 109.12404]

Try `warpAffine(smoothed,myPatch,warp_mat, patch.size(), WARP_INVERSE_MAP)` with `ofsx=0,ofsy=0`. — Quang Hoang, Apr 14 '17 at 09:40
@QuangHoang I've tried both by using `warp_mat.at(0,2) = 0` , `warp_mat.at(1,2) = 0` and the matrix is full now (no so many zeros), but the values are very differnet... — justHelloWorld, Apr 14 '17 at 10:10
Why don't you include `smoothed` as a picture and the size of `patch`. I'll try to play with it when I have time. — Quang Hoang, Apr 14 '17 at 10:12
@QuangHoang thanks for your interest. Smoothed is a 23x23 matrix, I posted an example of it. Patch is a 41x41 matrix. — justHelloWorld, Apr 14 '17 at 10:33
@QuangHoang could you please give a look at [this](http://stackoverflow.com/questions/43412722/replace-a-chain-of-image-blurs-with-one-blur) question too? — justHelloWorld, Apr 14 '17 at 16:00
@QuangHoang do you have any news on this? I really need this and I didn't find an equivalent function yet — justHelloWorld, May 03 '17 at 12:16

score 1 · Answer 1 · edited Jun 20 '20 at 09:12

1

I'm more familiar with this warpAffine, whose basic statement is

cv::warpAffine (InputArray src,     // input mat
                OutputArray dst,    // output mat
                InputArray M,       // affine transformation mat
                Size dsize)         // size of the output mat

where M is the matrix

      a11 a12 ofx
      a21 a22 ofy

In your term, the first two columns is the linear transformation matrix A, the last is the translation vector b.

The cv::hal::warpAffine() is just the same, where double M[6] corresponds to the above affine transformation matrix, but I'm not sure in which order it is flatten (most likely, [a11,a12,ofx,a21,a22,ofy]).

In OpenCV, the origin (0,0) is the top-left conner as usual, while in Intel's code, the origin (0,0) is in the middle of the image. That's what the part

for (int j=-halfHeight; j<=halfHeight; ++j)
{
   for(int i=-halfWidth; i<=halfWidth; ++i)
      {
         const int y = (int) floor(wy);
         //...
   }
}

does: (i,j) is the coordinate in res, j from -halfHeight to halfHeight and i from -halfHeight to halfHeight. So in this case (0,0) is in the center of the res image.

In the provided code, if you want to map src onto res (i guess), you would need to do:

    bool touchesBoundary = interpolate(smoothed, (float)(imageSize>>1), (float)(imageSize>>1), imageToPatchScale, 0, 0, imageToPatchScale, patch);

Notice here imageSize>>1 instead of patchImageSize>>1. Why? You want the center of the res (i=0,j=0) maps to the center of src, i.e. the value src.at<float>(src.cols/2, src.rows/2) (why?)

Now to make that work in your example, the equivalent of cv::warpedAffine() would be

warpAffine(smoothed, myPatch, warp_mat, patch.size(),WARP_INVERSE_MAP);

where the warp_mat has ofsx=0,ofsy=0.

Finally, here's an illustration of what I tried:

where diff = mypatch - patch >5 and smoothed is scaled up by OS. Notice the black border in patch, it is because the restrictions x < width and y<height in the code.

edited Jun 20 '20 at 09:12

Community

1
1

answered Apr 12 '17 at 09:14

Quang Hoang

146,074
10
56
74

Mmmh what do you mean by "origin"? And what if I use the Intel's code (you mean IPP, right)? That should be even more efficient, but I have no idea how to use it. – justHelloWorld Apr 12 '17 at 09:59
when you say _given a point `p=(x,y)`_, (x,y) is the coordinate of the point. For the given code, `(0,0)` lies in the center of the image. For OpenCV, `(0,0)` is the top-left corner. Also, I would use OpenCV's `warpAffine` since it may have some optimization underneath. – Quang Hoang Apr 12 '17 at 10:04
Oh so you're suggesting to apply `warpAffine` after having translated to this new coordinate system, right? What a mess, why the hell did they do this and not simply using the damned `warpAffine` from the beggining? – justHelloWorld Apr 12 '17 at 10:06
And btw the translation should be `newImage(0,0) = originalImage(halfHeight, halfWidth)` , `newImage(0,1) = originaImage(halfHeight, halfWidth+1)` and so on, right? – justHelloWorld Apr 12 '17 at 10:10
And btw I really still can't figure out how to do this translation that you're talking about, I'm sorry :( – justHelloWorld Apr 12 '17 at 10:19
The translation should get absorbed into the affine transform. In the original code, destination pixel (0,0) is computed with j and i equal to `-halfHeight` and `-halfWidth`. You need (0, 0) to give you the same result through your affine matrix, which is just algebra. Work it out carefully, but I think you need to use `ofx_new = ofx - a12*halfHeight - a11*halfWidth`, and similar for `ofy_new`. Use those for the third column of your matrix. – Peter Apr 12 '17 at 12:16
To help understand all this, all the matrix is doing is mapping your destination pixel coordinate (augmented with 1) back to the source coordinate. Write down the various operations, so that you have interpolated source coordinate equal to some function of destination coordinate. Then use algebra to "collect terms" so that it's of the form you need for your matrix: a11*dx + a22*dy + offx. You might need to take a brief detour through some basic linear algebra (definition of a matrix multiply for example) – Peter Apr 12 '17 at 12:19
@Peter Thanks for your comment. Actually the algebra here is not the problem (I'm quite confident with basic concepts like matrix multiplication), but my problem I think is that I'm not quite understanding what this code is doing. I'm sorry, I can really see that you're putting all your effort to make me understand this, but I think that your comment give as obvious a lot of image processing points. Btw, I tried to write a new version of the testing code in my question using your `ofx_new` and `ofy_new` from your last comment. Please give a look at it. Seriously, thanks for all the help. – justHelloWorld Apr 12 '17 at 19:18
Testing for exact equality is probably not the best metric of success. Rounding differences are possible. I would write out both images and compare visually, and inspect the difference image if they look about the same. – Peter Apr 13 '17 at 12:22
@Peter I'm sorry but using the method above `myPatch` results as a 0 matrix :( – justHelloWorld Apr 14 '17 at 08:46
@QuangHoang Could you please look at my updated answer and the comments here? I'm really stuck here and I don't know what to do – justHelloWorld Apr 14 '17 at 09:03
@Peter I'm sorry but using `ofx_new` and `ofy_new` as I've written above, `myPatch` results as a 0 matrix :( Btw I posted also the resulting matrix by using `ofx` and `ofy` instead of `ofx_new` and `ofy_new` – justHelloWorld Apr 14 '17 at 09:04

How can I rewrite this warp-affine using OpenCV?

1 Answers1

Linked