Augmented reality with Android sensors and OpenGL

Question

An image speaks always more than a ton of text, here's what I'm trying to do :

What is in the center of the circle is the user's phone position (origin). The app displays a custom camera view and it also shows an OpenGL scene (depending where you are looking). The OpenGL scene is only composed of a simple cube and when the user is looking in the right direction the cube is rendered.

I'm pretty new to OpenGL and I achieved to display the cube in front of the camera but it's static : I can't move around the 360° view.

I get from the sensors the orientation of the device :

    int type = event.sensor.getType();
    float[] data;
    if (type == Sensor.TYPE_ACCELEROMETER) {
        mGData = data;
    } else if (type == Sensor.TYPE_MAGNETIC_FIELD) {
        mMData = data;
    } else {
        // we should not be here.
        return;
    }
    for (int i=0 ; i<3 ; i++)
        data[i] = event.values[i];

    SensorManager.getRotationMatrix(mR, mI, mGData, mMData);
    SensorManager.getOrientation(mR, mOrientation);

As far as I understand, the 3 simultaneous orthogonal rotation angles are stored in mOrientation. But what then ? I wanted to make something like GLU.lookAt(0, 0, 0, X, Y, Z, ?, ?, ?) in the onDrawFrame method but it didn't work. I want to make something like that guy said he couldn't do (see the last paragraph here : https://stackoverflow.com/a/9114246/1304830).

Here's the code I used in onDrawFram :

    gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
    gl.glMatrixMode(GL10.GL_MODELVIEW);

    // Look in a direction (with the sensors)
    gl.glLoadIdentity();
    GLU.gluLookAt(gl, 0.0f, 0.0f, 0.0f, ?, ?, ?, ?, ?, ?); // Where I need help

    gl.glPushMatrix();  

    // Place the cube in the scene
    gl.glTranslatef(0, 0, -5);
    mCube.draw(gl);

Thanks for your help

Answer 1

This will do what that person was trying to do, but the problem with his idea is the up-vector is always pointing up on the y-axis. So if you roll the phone the camera isn't going to roll with it.

float pi = (float) Math.PI;
float rad2deg = 180/pi;

// Get the pitch, yaw and roll from the sensor. 

float yaw = orientation[0] * rad2deg;
float pitch = orientation[1] * rad2deg;
float roll = orientation[2] * rad2deg;

// Convert pitch, yaw and roll to a vector

float x = (float)(Math.cos( yaw ) * Math.cos( pitch ));
float y = (float)(Math.sin( yaw ) * Math.cos( pitch ));
float z = (float)(Math.sin( pitch ));

GLU.gluLookAt( gl, 0.0f, 0.0f, 0.0f, x, y, z, 0.0f, 1.0f, 0.0f );  

Using the three glRotates is a better option IMO, unless you want to lock the roll for some reason.

Note: I'm not sure which direction Android calls up in relation to the phone's screen so I may have got yaw, pitch and roll misconfigured.