OpenCV camera orientation issue

Question

I have a simple project that just show the camera with org.opencv.android.JavaCameraView.

my problem is that in default the camera is on landscape mode and I can't change this cause I need to define CameraBridgeViewBase instead of a regular camera intent.

this is a part of my code:

XML code:

        <LinearLayout
            android:layout_width="match_parent"
            android:layout_height="match_parent" >

            <org.opencv.android.JavaCameraView
                android:layout_width="fill_parent"
                android:layout_height="300dp"
                android:visibility="gone"
                android:id="@+id/HelloOpenCvView"
                opencv:show_fps="true"
                opencv:camera_id="1" />


        </LinearLayout>  

        <LinearLayout
            android:layout_width="match_parent"
            android:layout_height="wrap_content" >


            <Button
                android:id="@+id/BtnVideo"
                android:layout_marginLeft="2dp"
                android:layout_marginRight="2dp"                    
                android:layout_width="0dp"
                style="@style/button"
                android:layout_height="wrap_content"
                android:layout_weight="1.00"
                android:text="@string/videoBtn"
                android:textSize="18dip" />


        </LinearLayout>

Java Code :

 CameraBridgeViewBase mOpenCvCameraView;
    Button VideoButton;
 protected void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        overridePendingTransition(0, 0);

        VideoButton = (Button) this.findViewById(R.id.BtnVideo);

        VideoButton.setOnClickListener(onClickListener);

        mOpenCvCameraView= (CameraBridgeViewBase) findViewById(R.id.HelloOpenCvView);
        mOpenCvCameraView.setVisibility(SurfaceView.INVISIBLE);

    } 

        private OnClickListener onClickListener = new OnClickListener() {

            @Override
            public void onClick(View v) {
                    switch (v.getId()){

                        case R.id.BtnVideo:
                            if(mOpenCvCameraView.getVisibility() == SurfaceView.VISIBLE)
                            {
                                mOpenCvCameraView.setVisibility(SurfaceView.INVISIBLE);
                            }
                            else
                            {
                                mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
                            }

                            break;
                        default :
                            break;
                    }

            }
        };


        public void onResume() {
            super.onResume();
            overridePendingTransition(0, 0);
            OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_3, this, mLoaderCallback);
        }
         public void onPause()
         {
             super.onPause();
             if (mOpenCvCameraView != null)
                 mOpenCvCameraView.disableView();
         }
         public void onDestroy() {
             super.onDestroy();
             if (mOpenCvCameraView != null)
                 mOpenCvCameraView.disableView();
         }
         public void onCameraViewStarted(int width, int height) {
         }

         public void onCameraViewStopped() {
         }
         public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
             return inputFrame.rgba();
         }

        private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
            @Override
            public void onManagerConnected(int status) {
                switch (status) {
                    case LoaderCallbackInterface.SUCCESS:
                    {
                        //Log.i(TAG, "OpenCV loaded successfully");
                        mOpenCvCameraView.enableView();
                    } break;
                    default:
                    {
                        super.onManagerConnected(status);
                    } break;
                }
            }
        };

So how can I change the default orientation?

Thanks!

score 41 · Accepted Answer · edited Oct 24 '18 at 06:37

41

Ok, I found this as a solution:

First I get into JavaCameraView.java class in the OpenCV Library - 2.4.5

and then in initializeCamera() function before mCamera.startPreview(); I added these 2 function:

            setDisplayOrientation(mCamera, 90);
            mCamera.setPreviewDisplay(getHolder());

and the first function implemented like this:

protected void setDisplayOrientation(Camera camera, int angle){
    Method downPolymorphic;
    try
    {
        downPolymorphic = camera.getClass().getMethod("setDisplayOrientation", new Class[] { int.class });
        if (downPolymorphic != null)
            downPolymorphic.invoke(camera, new Object[] { angle });
    }
    catch (Exception e1)
    {
        e1.printStackTrace();
    }
}

I just reminding that I work with OpenCV.

Hope this help someone.

edited Oct 24 '18 at 06:37

Akshay Paliwal

3,718
2
39
43

answered May 22 '13 at 06:00

user2235615

1,513
3
17
37

4

After using your code, it works perfectly fine. However, in my actual android program, I have some code inside onCameraFrame to modify the RGBA output from the camera. The display somehow remains unmodified on the screen. Do you know the reason for that? I tried to save the modified MAT, which looks fine in the saved file, but just the preview on screen is wrong. – John Yang Oct 10 '13 at 14:43
2

I found another solution you can see it here: http://answers.opencv.org/question/20325/how-can-i-change-orientation-without-ruin-camera/ – user2235615 Oct 20 '13 at 07:47
Any other solution because above code not working for OpenCV Library - 2.4.9 – Mukesh Parmar Jun 13 '14 at 11:25
1

You can use my solution that works for every OpenCV version : **http://answers.opencv.org/question/20325/how-can-i-change-orientation-without-ruin-camera/** – user2235615 Jun 15 '14 at 05:39
1

It is giving error that it seems that your phone doesnot support camera or is blocked! – Abid Dec 09 '14 at 08:07
@Abid, That's not an OpenCV-related issue. Add camera permission to Manifest. – Jeon Jan 16 '16 at 07:42
3

@user2235615 your code is working but the problem is that the frame is not in the full screen in portrait orientation and looking too much stretched (wider) or it is not working in landscape orientation is there any another solution – UltimateDevil Jun 13 '17 at 13:15

Thong Doan · Answer 2 · 2018-11-16T04:01:45.890

I am using OpenCV 3.1, I fix it by apply transform when draw bitmap on deliverAndDrawFrame method of CameraBridgeViewBase class, Hope it helpful:

On CameraBridgeViewBase.java :

//I added new field
private final Matrix mMatrix = new Matrix();

//added updateMatrix method 
private void updateMatrix() {
    float hw = this.getWidth() / 2.0f;
    float hh = this.getHeight() / 2.0f;
    boolean isFrontCamera = Camera.CameraInfo.CAMERA_FACING_FRONT == mCameraIndex;
    mMatrix.reset();
    if (isFrontCamera) {
        mMatrix.preScale(-1, 1, hw, hh);
    }
    mMatrix.preTranslate(hw, hh);
    if (isFrontCamera)
        mMatrix.preRotate(270);
    else
        mMatrix.preRotate(90);
    mMatrix.preTranslate(-hw, -hh);
}

//then We need call updateMatrix on layout
@Override
public void layout(int l, int t, int r, int b) {
    super.layout(l, t, r, b);
    updateMatrix();
}

//I think we should also call updateMatrix on measure
@Override
protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
    super.onMeasure(widthMeasureSpec, heightMeasureSpec);
    updateMatrix();
}


//then We need update deliverAndDrawFrame
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
    //....Origin Code...

    //Set matrix before OpenCV draw bitmap
    int saveCount = canvas.save();
    canvas.setMatrix(mMatrix);

    //Begin OpenCv origin source
    if (mScale != 0) {
        canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
             new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
             (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
             (int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
             (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
    } else {
         canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
             new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
             (canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
             (canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
             (canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
    }
    //End OpenCv origin source

    //Restore canvas after draw bitmap
    canvas.restoreToCount(saveCount);

    //....Origin code...
}



//After that We can see that the camera preview is so small, the easiest way is change mScale Value (should we change mScale to "private" instead "protected" ?)
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
    //....Origin Code...

    //Set matrix before OpenCV draw bitmap to screen
    int saveCount = canvas.save();
    canvas.setMatrix(mMatrix);


    //Change mScale to "Aspect to fill"
    mScale = Math.max((float) canvas.getHeight() / mCacheBitmap.getWidth(), (float) canvas.getWidth() / mCacheBitmap.getHeight());

    //Begin OpenCv origin source
    if (mScale != 0) {
        canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
             new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
             (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
             (int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
             (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
    } else {
         canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
             new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
             (canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
             (canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
             (canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
    }
    //End OpenCv origin source

    //Restore canvas after draw bitmap
    canvas.restoreToCount(saveCount);

    //....Origin Code...
}

You can get full source code here: https://gist.github.com/thongdoan/d73267eb58863f70c77d1288fe5cd3a4

Hi, thanks for your contribution. Instead of pasting a whole class, I think it'd be more useful to indicate just the changes you have done. — Jose Gómez, Feb 14 '17 at 00:00
[This answer is being discussed on meta.](https://meta.stackoverflow.com/questions/376637/edit-rejected-because-reason-for-edit-misunderstood-how-to-proceed) — Script47, Nov 14 '18 at 11:10
this worked, but doesn't scale the camera to fit the screen for me. — mheavers, Feb 23 '19 at 00:03
@Jesús - It's been awhile - I figured out a solution that worked for me and wrote a little tutorial on it here: https://heartbeat.fritz.ai/working-with-the-opencv-camera-for-android-rotating-orienting-and-scaling-c7006c3e1916 — mheavers, Jan 15 '20 at 05:26

Ramón Pérez Silva · Answer 3 · 2016-09-20T17:16:07.937

The problem is that the code that paints doesn't check the camera params. The Mat is drawn on the Surface view in the function "deliverAndDrawFrame" in the class "CameraBridgeViewBase".

With a very simple modification in the CameraBridgeViewBase class, we can make a function that rotates the way the bitmap is drawn.

int userRotation= 0;

public void setUserRotation(int userRotation) {
    this.userRotation = userRotation;
}

/**
 * This method shall be called by the subclasses when they have valid
 * object and want it to be delivered to external client (via callback) and
 * then displayed on the screen.
 * @param frame - the current frame to be delivered
 */
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
    Mat modified;

    if (mListener != null) {
        modified = mListener.onCameraFrame(frame);
    } else {
        modified = frame.rgba();
    }

    boolean bmpValid = true;
    if (modified != null) {
        try {
            Utils.matToBitmap(modified, mCacheBitmap);
        } catch(Exception e) {
            Log.e(TAG, "Mat type: " + modified);
            Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
            Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
            bmpValid = false;
        }
    }

    if (bmpValid && mCacheBitmap != null) {
        Canvas canvas = getHolder().lockCanvas();
        if (canvas != null) {
            canvas.drawColor(Color.parseColor("#8BC34A"), PorterDuff.Mode.SRC_IN);
 //this is the rotation part
            canvas.save();
            canvas.rotate(userRotation,  (canvas.getWidth()/ 2),(canvas.getHeight()/ 2));

            if (mScale != 0) {
                canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
                     new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
                     (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
                     (int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
                     (int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
            } else {
                 canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
                     new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
                     (canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
                     (canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
                     (canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
            }

            if (mFpsMeter != null) {
                mFpsMeter.measure();
                mFpsMeter.draw(canvas, 20, 30);
            }
//remember to restore the canvas 
            canvas.restore();
            getHolder().unlockCanvasAndPost(canvas);
        }
    }
}

I tried the most common solution that use the Core.flip function that rotates the Mat but consumes a lot of resources, this solution doesn't affect to the detection and doesn't affect to the performance, only changes the way the image is drawn on the canvas.

Hope this help.

This worked great for me, but won't account for any resizing of the canvas that needs to happen after rotation — mheavers, Feb 22 '19 at 23:26

score 9 · Answer 4 · edited Jul 20 '22 at 09:20

9

Try this one on your onCameraFrame

mRgba = inputFrame.rgba();
Mat mRgbaT = mRgba.t();
Core.flip(mRgbaT, matRgbaFlip, 1);
Imgproc.resize(matRgbaFlip, matRgbaFlip, mRgba.size());
mRgbaT.release();
return matRgbaFlip;

EDIT: Fix memory leak, releasing the Mat created in the method and moving the other one as a global variable.

edited Jul 20 '22 at 09:20

Edgar Alvarado

57
7

answered Dec 01 '16 at 04:04

Android Mediocre

310
3
14

3

This works but it closes the app after a few sec. Not sure why, don't see any fatal errors in the console. – Abhinav Raja Sep 22 '17 at 02:34
1

app crash is probably about memory leak. have to release mRgba before returning. And plus, creating mRgbaT as global variable solved the problem. – donmezburak Oct 23 '18 at 15:57

score 1 · Answer 5 · answered Jan 19 '19 at 22:42

Firstly, do not create an instance from the base class, instead get the instance from the extended class

//CameraBridgeViewBase mOpenCvCameraView;
JavaCameraView mOpenCvCameraView;

It is worth mentioning that the CameraBridgeViewBase.java has already a surface holder, so let's use it instead of creating a surface texture.

Hence secondly, edit the JavaCameraView.java inside the function initializeCamera() by replacing the surface texture with a surface holder

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) {

    //mSurfaceTexture = new SurfaceTexture(MAGIC_TEXTURE_ID);
    //mCamera.setPreviewTexture(mSurfaceTexture);

    mCamera.setPreviewDisplay(getHolder());

} else
    mCamera.setPreviewDisplay(null);

The last step is to set the orientation WITHOUT adding any special functions. In the same function initializeCamera() just before startPreview() call setDisplayOrientation(degrees)

/* Finally we are ready to start the preview */
Log.d(TAG, "startPreview");
mCamera.setDisplayOrientation(90);
mCamera.startPreview();

This works, but it stops the object detector from working. – qed Aug 03 '19 at 18:45 — qed, Aug 03 '19 at 18:45

score 0 · Answer 6 · answered Nov 09 '20 at 12:16

0

In OpenCV version 3.4.3

I fixed it by doing below changes in initializeCamera() method in JavaCameraView.java class.

setDisplayOrientation(mCamera, 0);

Steps to follow:

Go to the JavaCameraView.java file.
Search for "setDisplayOrientation"
Set orientation angle to 0 (By default it was 90).

answered Nov 09 '20 at 12:16

Sameer Shelar

118
6

didn't work for me in 3.4.16 – Rohaitas Tanoli Jul 27 '23 at 12:22

score -9 · Answer 7 · answered Dec 02 '14 at 12:19

-9

the android:screenOrientation value in the AndroidManifest.xml should help.

android:screenOrientation="portrait"

answered Dec 02 '14 at 12:19

patel deven

630
7
9

4

He asks about the orientation of the camera, not the screen. – Bennyz Dec 03 '14 at 17:50
i too had the same problem, that the camera used to be in portrait mode. i had to change the orientation manually in order to set things right – patel deven Dec 19 '14 at 01:39

OpenCV camera orientation issue

7 Answers7

Linked