2

I am trying to do video stabilization with opencv(without the opencv video stabilization class).

the steps for my algo is as follows->

  1. Surf points extraction,

  2. Matching,

  3. Homography matrix,

  4. warpPerspective

And the output video is not stabilized at all :(. it just looks like the original video. I could not find and reference code for video stabilization. I followed the procedure described here . Can anybody help me out by telling me where I am going wrong or provide me some source code link to improve my algo.

Please help. Thank you

Community
  • 1
  • 1
MMH
  • 1,676
  • 5
  • 26
  • 43
  • This question is both very vague, and very broad. You will need to rephrase it and give us more information if we are to be able to help you. – Hannes Ovrén Feb 07 '14 at 10:32

3 Answers3

3

You can use my code snippet as a start point (not very stable but seems it works):

#include "opencv2/opencv.hpp"
#include <iostream>
#include <vector>
#include <stdio.h>

using namespace cv;
using namespace std;

int main(int ac, char** av)
{
    VideoCapture capture(0);
    namedWindow("Cam");
    namedWindow("Camw");
    Mat frame;
    Mat frame_edg;
    Mat prev_frame;
    int k=0;
    Mat Transform;
    Mat Transform_avg=Mat::eye(2,3,CV_64FC1);
    Mat warped;
    while(k!=27)
    {
        capture >> frame;
        cv::cvtColor(frame,frame,cv::COLOR_BGR2GRAY);
        cv::equalizeHist(frame,frame);
        cv::Canny(frame,frame_edg,64,64);
        //frame=frame_edg.clone();
        imshow("Cam_e",frame_edg);
        imshow("Cam",frame);

        if(!prev_frame.empty())
        {
            Transform=estimateRigidTransform(frame,prev_frame,0);
            Transform(Range(0,2),Range(0,2))=Mat::eye(2,2,CV_64FC1);
            Transform_avg+=(Transform-Transform_avg)/2.0;
            warpAffine(frame,warped,Transform_avg,Size( frame.cols, frame.rows));

            imshow("Camw",warped);
        }

        if(prev_frame.empty())
        {
            prev_frame=frame.clone();
        }

        k=waitKey(20);      
    }
    cv::destroyAllWindows();
    return 0;
}

You can also look for paper: Chen_Halawa_Pang_FastVideoStabilization.pdf as I remeber there was MATLAB source code supplied.

Andrey Smorodov
  • 10,649
  • 2
  • 35
  • 42
  • Hi Andrey Thanks for your reply. can you please explain what does " Transform(Range(0,2),Range(0,2))=Mat::eye(2,2,CV_64FC1); Transform_avg+=(Transform-Transform_avg)/2.0;" this 2 lines does? And also I saw the matlab code, I am trying to convert it in C++. – MMH Feb 10 '14 at 02:14
  • 1
    The Transform(Range(0,2),Range(0,2))=Mat::eye(2,2,CV_64FC1); line discardes all transformations (rotation,scale and shear) except translations. This was done as I didn't need this transform compensation and it increase stability of algorithm. The line Transform_avg+=(Transform-Transform_avg)/2.0; it's the difference for transform displacement compensation. 1/2 it's coefficient for mor robustness as I remember (I wrote this code a couple years ago) it works also as low pass filter. – Andrey Smorodov Feb 10 '14 at 07:52
  • Thank you Andrew, I really appreciate your answer. But my test video doesn't give good result with the code snipt you provided. thanks anyways. – MMH Feb 11 '14 at 04:28
2

In your "warpAffine(frame,warped,Transform_avg,Size( frame.cols, frame.rows));" function, you must specify FLAG as WARP_INVERSE_MAP for stabilization.

Sample code I have written:

Mat src, prev, curr, rigid_mat, dst;

VideoCapture cap("test_a3.avi");

while (1)
{
    bool bSuccess = cap.read(src);
    if (!bSuccess) //if not success, break loop
        {
                cout << "Cannot read the frame from video file" << endl;
                break;
        }

    cvtColor(src, curr, CV_BGR2GRAY);

    if (prev.empty())
    {
        prev = curr.clone();
    }

    rigid_mat = estimateRigidTransform(prev, curr, false);

    warpAffine(src, dst, rigid_mat, src.size(), INTER_NEAREST|WARP_INVERSE_MAP, BORDER_CONSTANT);


    // ---------------------------------------------------------------------------//

    imshow("input", src);
    imshow("output", dst);

    Mat dst_gray;
    cvtColor(dst, dst_gray, CV_BGR2GRAY);
    prev = dst_gray.clone();

    waitKey(30);
}

Hoping this will solve your problem :)

user3566188
  • 115
  • 5
0

Surf is not so fast. the way I work is with Optical Flow. First you have to calculating good features on your first frame with the GoodFeaturesToTrack() function. After that I do some optimalisation with the FindCornerSubPix() function.

now you have the featurepoints in your startframe, the next thing you have to do is determine the optical flow. There are several Optical Flow functions but the one I use is OpticalFlow.PyrLK(), in one of the out parameters you get the featurespoints in the current frame. With that you can calculate the Homography matrix with the FindHomography() function. Next you have to do is invert this matrix, the explanation you can easily find with google, next you call the WarpPerspective() function to stabilize your frame.

PS. The functions I put here where from EmguCV, the .NET wrapper for OpenCV, so ther may be some differences

hof_mat
  • 1