11

I'm trying to implement ZCA Whitening algorithm like shown here: How to implement ZCA Whitening? Python with opencv in Scala (using Java api) but I cannot find the most of the functions used there (python with numpy).

So far I tried with this:

Loading image as Mat:

val input = Imgcodecs.imread(img)
Imgproc.cvtColor(input, input, Imgproc.COLOR_BGR2GRAY)
input.convertTo(input, CvType.CV_64FC1)

Then apply the algorithm:

//Covariance matrix
val covar, mean = new Mat()
Core.calcCovarMatrix(input, covar, mean, Core.COVAR_NORMAL | Core.COVAR_ROWS) 
Core.divide(covar, new Scalar(input.rows - 1), covar)

//Singular Value Decomposition
val w, u, vt = new Mat()
Core.SVDecomp(covar, w, u, vt, Core.SVD_FULL_UV)

//#Whitening constant, it prevents division by zero
val  epsilon = 1e-5

To implement the last trasformation

ZCAMatrix = np.dot(U, np.dot(np.diag(1.0/np.sqrt(S + epsilon)), U.T))

I tried with:

var ZCAMatrix = new Mat
Core.add(w, new Scalar(epsilon), ZCAMatrix)
Core.sqrt(ZCAMatrix, ZCAMatrix)
Core.divide(1.0, ZCAMatrix, ZCAMatrix)
Core.gemm(Mat.diag(ZCAMatrix), u.t, 1, new Mat, 0, ZCAMatrix)
Core.gemm(u, ZCAMatrix, 1, new Mat, 0, ZCAMatrix)

Then apply the trasformation to the original image:

val output = new Mat
Core.gemm(ZCAMatrix, input, 1, new Mat, 0, output)

The result of thei transformation is: this

which is not exactly what it's supposed to be. Can someone help?

ascripter
  • 5,665
  • 12
  • 45
  • 68
drstein
  • 1,113
  • 1
  • 10
  • 24
  • 1
    In matrix multiplication there is importance in the order you multiply so i think you should use Core.gemm(u, ZCAMatrix, 1, new Mat, 0, ZCAMatrix) – Amitay Nachmani Sep 08 '16 at 15:46
  • you're right! tried with that, the result changes but it's still wrong. – drstein Sep 09 '16 at 11:02
  • 1
    @drstein it could be due to SVDecomp not returning non-square u (need to pass FULL_UV), while numpy by default return the full matrix. this is just a guess, I haven't tried to compile this. an easier way, perhaps, is to compute pca using opencv first then convert to zca. – fireant Sep 10 '16 at 04:13
  • Thanks, I updated the question with both suggestions, with the way I load the picture and with the new result. – drstein Sep 12 '16 at 09:55
  • 1
    @drstein, maybe it'll be easier to translate a C++ code into Java? You can look on the first section of the code here: http://www.compvision.ru/forum/index.php?/topic/991-%D0%BD%D0%BE%D1%80%D0%BC%D0%B0%D0%BB%D0%B8%D0%B7%D0%B0%D1%86%D0%B8%D1%8F-%D0%B8%D0%B7%D0%BE%D0%B1%D1%80%D0%B0%D0%B6%D0%B5%D0%BD%D0%B8%D1%8F-%D0%BF%D1%80%D0%B8%D0%B2%D0%B5%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5-%D0%BA-%D1%84%D0%BE%D1%80%D0%BC%D0%B5-%D1%83%D0%B4%D0%BE%D0%B1%D0%BD%D0%BE%D0%B9-%D0%B4%D0%BB%D1%8F-%D0%BA%D0%BB%D0%B0%D1%81%D1%81%D0%B8%D1%84%D0%B8%D0%BA%D0%B0%D1%82%D0%BE%D1%80%D0%BE%D0%B2/ – Andrei K. Apr 25 '17 at 07:57
  • Does `val covar, mean = new Mat()` mean that you get two distinct new `Mat`s or that you get two references **to the same** `Mat` ? – Pibben Oct 18 '19 at 11:06

0 Answers0