2

I was struggling to maximize Tr(W'X'XW) with constraint W'W = I in TensorFlow. The problematic part is the constraint since it requires the weights stay on a Stiefel manifold. There is a soft way which adds the constraint as a Lagrangian term to the objective. However, this is not exact, and implementation can be messy.

I'm aware of the Pymanopt library that uses autodiff for manifold optimization. But I am still wondering if there was a principle way to simulate (Stiefel) manifold optimization within Tensorflow?

Klinux
  • 21
  • 2
  • I don't know much about Tensorflow, but the optimization problem you're trying to solve is equivalent to finding the mode of a matrix Bingham distribution, which is straightforward to compute. Suppose W is p x k dimensional and thus X'X is p x p dimensional. If I recall correctly, the maximizing W_max is constructed from the first k eigenvectors of X'X, i.e. if we let X'X = UDU' be the eigendecomposition of X'X, then W_max = U[:,1:k]. There is one other maximizing matrix: -W_max. – user18297 Jul 16 '17 at 23:23

0 Answers0