How to extract the eye/target/up from a view matrix using sympy?

Question

We've tried to figure out for quite a while in the #python channel how to compute the eye/target/up vectors out of a view matrix using sympy. One possible way to do it could be:

from sympy import *
from pprint import pprint

v1, v2, v3, v4 = symbols('v1 v2 v3 v4')
v5, v6, v7, v8 = symbols('v5 v6 v7 v8')
v9, v10, v11, v12 = symbols('v9 v10 v11 v12')
v13, v14, v15, v16 = symbols('v13 v14 v15 v16')

V = Matrix([
    [v1, v2, v3, v4],
    [v5, v6, v7, v8],
    [v9, v10, v11, v12],
    [v13, v14, v15, v16],
    ])
u1, u2, u3 = symbols('u1 u2 u3', real=True)
t1, t2, t3 = symbols('t1 t2 t3', real=True)
e1, e2, e3 = symbols('e1 e2 e3', real=True)

U = Matrix([u1, u2, u3])
T = Matrix([t1, t2, t2])
E = Matrix([e1, e2, e3])

def calculate_view_matrix(up, eye, target):
    zaxis = (eye - target).normalized()
    xaxis = up.cross(zaxis).normalized()
    yaxis = zaxis.cross(xaxis)

    orientation = Matrix([
        [xaxis[0], yaxis[0], zaxis[0], 0],
        [xaxis[1], yaxis[1], zaxis[1], 0],
        [xaxis[2], yaxis[2], zaxis[2], 0],
        [0, 0, 0, 1],
            ])

    translation = Matrix([
            [1, 0, 0, 0],
            [0, 1, 0, 0],
            [0, 0, 1, 0],
            [-eye[0], -eye[1], -eye[2], 1],
            ])

    return orientation * translation

print(V - calculate_view_matrix(U, E, T))

s = solve([
    V - calculate_view_matrix(U, E, T),
    U.norm() - 1,
    T.norm() - 1],
    [u1, u2, u3, t1, t2, t3, e1, e2, e3])

print(s)

But for some reason that script has been running for ~20 minutes and sympy hasn't been able to give any solution so far.

Another attempt has also been trying to simplify the above generic problem to something simpler as how would you compute the up vector?

In a simpler context, the problem definition would be something like this:

1. u,z,x are 3d vectors which forms an orthonormal basis.
2. z, x are constant vectors
3. u is the unknown vector

And the equation to solve this:

u.cross(z).normalized() - x

If you tried to solve a simple particular case of the above generic equation like this...

from sympy import *
u1,u2,u3=symbols('u1 u2 u3', real = True)
x=Matrix([1,0,0])
z=Matrix([0,0,1])
u=Matrix([u1,u2,u3])

print(solve(u.cross(z).normalized() - x, u))

you'd get NotImplementedError: could not solve u2 - Abs(u2).

NS: Thing is, in order to extract the inputs from the view matrix is required than the function computing the matrix be injective or bijective, otherwise the initial information will be lost. If you don't add any constraints the above functions are definitely not injectives because at the moment which is using a normalize operation the function becomes automatically not injective anymore, for instance:

a) normalize(x) = x/|x|
b) To prove normalize is injective then normalize(a)=normalize(b) should give a=b
c) normalize(a)=normalize(b) => a/|a|=b/|b| , which is not true then normalize is not injective

Of course, this could be trivially proved just saying than infinitelly vectors can provide the same normalized vector.

That's the reason why there has been added few constraints to calculate_view_matrix. Ie: U.norm() - 1, T.norm() - 1. Theorically, that should grant the calculate_view_matrix to become injective... (or not :))

So the main question would be, how can you constrain/modify properly the calculate_view_matrix so it can calculate the eye/target/up vectors out of the view matrix?

Answer 1

Besides a typo (T = Matrix([t1, t2, t2])) there are several flaws in your Ansatz of getting up, eye and target vectors back from a view matrix:

• The view matrix describes a rigid transformation in 3D, which has only 6 degrees of freedom (3 axis rotations, 3 axis translations). This roughly means that only 6 out of the 16 values v1, v2, ..., v16 can be chosen (more or less arbitrarily), the others are dependent or determined any way (e.g. v4 = v8 = v12 = 0, v16 = 1, v3**2 = 1 - v1**2 - v2**2, ...). So in general the equations from the matrix difference are contradicting.
• Even when requiring U.norm() - 1 = 0 the up-vector U can take infinitely many values (one angle is not determined). To reduce the possible solutions for U to finitely many cases one could add the condition U*(E-T) = 0.
• The condition T.norm() - 1 = 0 is wrong. It is T - E (the direction of view) that could/should be requested to have length 1.

Altogether I do not see a way to fix the Ansatz s.t. U, E, T could be computed from the view matrix using equations and sympy. But U, E, T can be easily extracted from the view matrix:

• a normalized U (fulfilling the above requirements) can be read from the second column
• -E can be read from the last row
• a normalized view vector E - T can be read from the third column

In sympy/Python code:

def calculate_up_eye_target(viewMat):
  eye = -viewMat[3,0:3].T
  target = eye - viewMat[0:3,2]
  up = viewMat[0:3,1]
  return up, eye, target