OpenCV FlannBasedMatcher gives different result when using for one pair image matching and when using for couple of pair image matching

Question

I want to find a template image on some scene images. So I try to find Homography between scene images and template image. I'm using OpenCV and c++. First using SURF to find descriptors for image and template. Then using FlannBasedMatcher to get matches point. everything seemed to be good, but when I run this program just for each image lonely, result is different with when I run it for all scene images.

Ptr<FlannBasedMatcher> matcher = FlannBasedMatcher::create();
vector<DMatch> matches;
matcher->match(objectDescriptor, sceneDescriptor, matches);
double max_dist = 0, min_dist = DBL_MAX;

for (int i = 0; i < objectDescriptor.rows; i++)
{
    double dist = matches[i].distance;
    if (dist < min_dist) min_dist = dist;
    if (dist > max_dist) max_dist = dist;
}

vector< DMatch > goodMatches;
for (int i = 0; i < objectDescriptor.rows; i++)
{
    if (matches[i].distance < max((coeffGoodMatches * min_dist), maxDistanceMatches))
    {
        goodMatches.push_back(matches[i]);
    }
}

vector<Point2f> obj;
vector<Point2f> scene;
for (int i = 0; i < goodMatches.size(); i++)
{
    //-- Get the keypoints from the good matches
    obj.push_back(keypoints_object[goodMatches[i].queryIdx].pt);
    scene.push_back(keypoints_scene[goodMatches[i].trainIdx].pt);
}
Mat H = findHomography(obj, scene, CV_RANSAC);

Mat result;
warpPerspective(img_scene, result, H, img_object.size(), WARP_INVERSE_MAP);

The key points and descriptors which find by SURF is same for each scenario, but when FlannBasedMatcher try to match descriptors, result is different. I also try to clear FlannBasedMatcher like below, but result is the same without clear.

matcher->clear();

Any idea?