Feature Whitening generates same image

Question

I am looking at the feature whitening example shown here: How to implement ZCA Whitening? Python

I have implemented the code below but whenever I run it I get the same image in all 4 pictures and I am not sure why:

Code to read data (works fine)

import cv2
import matplotlib.pyplot as plt
from urllib.request import urlopen, Request
new_size = 50

urls = ['https://hips.hearstapps.com/hmg-prod.s3.amazonaws.com/images/dog-puppy-on-garden-royalty-free-image-1586966191.jpg',
        'https://dogtime.com/assets/uploads/2018/10/puppies-cover.jpg',
         'https://www.readersdigest.ca/wp-content/uploads/2013/03/6-facts-to-know-before-owning-a-puppy.jpg',
          'https://images.newscientist.com/wp-content/uploads/2021/06/03141753/03-june_puppies.jpg']

puppies = []
for url in urls:
    r = Request(url, headers={'User-Agent': 'Mozilla/5.0'})
    response = urlopen(r).read()
    arr = np.asarray(bytearray(response), dtype=np.uint8)
    puppies.append(cv2.imdecode(arr, 0)) # 'Load it as it is'

#item_pics = [cv2.imread( './pictures/picture_'+ str(x) + '.jpeg', 0) for x in range(1,5)]
puppies_scaled = [cv2.resize(x, [new_size,new_size], interpolation = cv2.INTER_AREA) for x in puppies]
puppies_vectors = [x.reshape(new_size * new_size) for x in puppies_scaled]

Code to do the Whitening

#borrowed heavily from  https://stackoverflow.com/questions/31528800/how-to-implement-zca-whitening-python
def zca_whitening_matrix(X):
    """
    Function to compute ZCA whitening matrix (aka Mahalanobis whitening).
    INPUT:  X: [M x N] matrix.
        Rows: Variables
        Columns: Observations
    OUTPUT: ZCAMatrix: [M x M] matrix
    """
    # Covariance matrix [column-wise variables]: Sigma = (X-mu)' * (X-mu) / N
    sigma = np.cov(X, rowvar=True) # [M x M]
    # Singular Value Decomposition. X = U * np.diag(S) * V
    U,S,V = np.linalg.svd(sigma)
        # U: [M x M] eigenvectors of sigma.
        # S: [M x 1] eigenvalues of sigma.
        # V: [M x M] transpose of U
    # Whitening constant: prevents division by zero
    epsilon = 1e-25
    # ZCA Whitening matrix: U * Lambda * U'
    ZCAMatrix = np.dot(U, np.dot(np.diag(1.0/np.sqrt(S + epsilon)), U.T)) # [M x M]
    return ZCAMatrix

Code to Whiten

combined_data = np.array(puppies_vectors).T
my_zca_transformer_image_stuff = zca_whitening_matrix(combined_data)
xZCAMatrix = np.dot(my_zca_transformer_image_stuff, combined_data)
reshaped_pics = [xZCAMatrix[:,i].reshape(new_size,new_size) for i in range(4)]

Result (all pictures the same)

score 1 · Accepted Answer · answered Feb 09 '22 at 07:57

Whitening is performed on zero-mean data. This means, that the matrices are computed on the zero-mean data and that they should also be applied on the zero-mean data. If you don't do that, you will essentially see the transformed mean vector W*mean_imagein all images. If that dominates in magnitude, the pictures all look identical due to rounding effects. However, they aren't if you compute the differences between your images.

In code, you can see all that with

combined_data = np.array(puppies_vectors).T
my_zca_transformer_image_stuff = zca_whitening_matrix(combined_data)
mean_image = np.mean(combined_data,axis=1)
without_mean = (combined_data-mean_image[:,None])
xZCAMatrix = np.dot(my_zca_transformer_image_stuff, without_mean)
reshaped_pics = [xZCAMatrix[:,i].reshape(new_size,new_size) for i in range(4)]
for i in reshaped_pics:
    plt.imshow(i, cmap='gray')
    plt.show() # Should look good now
## What you essential saw is this transformed mean image
i = np.dot(my_zca_transformer_image_stuff, mean_image).reshape(new_size,new_size)
plt.imshow(i, cmap='gray')
plt.show()

Yikes! You're right. I typically have a preprocessing step built in that scales and centers but forgot to do that here. Thank you! — user1357015, Feb 09 '22 at 15:02

Feature Whitening generates same image

1 Answers1