Android - Moving view according to sensor rotation matrix

Question

Preface: I'm very new to graphics.

I'm trying to learn a bit more about AR and have a pet project that should allow me to explore my location for AR objects. effectively I want to stick a flat image in the real world and use my phone's sensors to move it such that it is only visible if I'm looking at it.

I followed the examples on the Android Developers page and I have both a rotation matrix and orientation. Both seem to make sense. I have my image which for now I just want to always fix at North but will eventually randomize it to be X degrees off North as well as some up/down etc.

I'm lost as to how to place my image in 3D space and how to use the rotation vector to determine the image's position on the screen.

Any help or links to articles would be greately appreciated.

I'm using the sample code from Android's developer's page:

@Override
public void onSensorChanged(SensorEvent event) {
    if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
        mGravity = event.values;
    } else if (event.sensor.getType() == Sensor.TYPE_MAGNETIC_FIELD) {
        mGeomagnetic = event.values;
    }
    if (mGravity == null || mGeomagnetic == null) {
        return;
    }

    float R[] = new float[16];
    float I[] = new float[9];
    if (SensorManager.getRotationMatrix(R, I, mGravity, mGeomagnetic)) {
        Log.d(TAG, ...);
    }
}

My gut instinct was to take the Matrix.invertM of R and then use Matrix.multiplyMV multiplying the inverted rotation matrix by the world position of object I want to render (for now lets assume the object is 5m north of me putting it at (0, 5, 0, 0)). I used a compass and a level to make sure my surface is flat and my phone is oriented N. The result of the print statement above is the following rotation matrix:

-0.37,  0.93,  0.02,  0.00
-0.92, -0.37,  0.07,  0.00
 0.07,  0.01,  1.00,  0.00
 0.00,  0.00,  0.00,  1.00

This seems wrong as the matrix should be the identity matrix:

1, 0, 0, 0
0, 1, 0, 0
0, 0, 1, 0
0, 0, 0, 1

score 0 · Accepted Answer · edited May 23 '17 at 12:33

Rather then trying to see where you are wrong, I would highly recommend you to try and get into it yourself. There are samples and answered questions on the web that show exactly what you want, so obviously you can try them and learn from them (one example: Android sensors for OpenGL, there are also some on github).

Also, since you have said you are very new to graphics, I would suggest to first do some heavy reading on matrices and transformations and understanding of the 3D space.

Examples: http://www.songho.ca/opengl/gl_transform.html https://www.raywenderlich.com/3664/opengl-tutorial-for-ios-opengl-es-2-0

Android - Moving view according to sensor rotation matrix

1 Answers1