Rotating a reshaped image as a matrix operation

Question

I have a gray scale image that I want to rotate. However, I need to do optimization on it. Therefore, I cannot use pillow or opencv. I want to reshape this image using python with numpy.reshape into an one dimensional vector (where I use the default settings C-style reshape). And thereafter, I want to rotate this image around a point using matrix multiplication and addition, i.e. it should be something like

rotated_image_vector = A @ vector + b # (or the equivalent in homogenious coordinates).

After this operation I want to reshape the outcome back to two dimensions and have the rotated image. It would be best if it would as well use linear interpolation between the pixels that do not fit exactly to an other pixel.

The mathematical theory tells it is possible, and I believe there is a very elegant solution to this problem, but I do not see how to create this matrix. Did anyone already have this problem or sees an immediate solution?

Thanks a lot, Eike

Sorry, but your idea is a little naive. Image rotation is such a common operation that optimized solutions have been studied since long. Working with Python rather than OpenCV cannot lead you to an improvement. OpenCV is written in C++, hence compiled, and uses hardware acceleration where possible. Last but not least, `v' = A @ v + b` would take a huge matrix `A` of size N² x N², essentially empty and impossible to store as dense, and would be totally inefficient. — , Sep 01 '21 at 09:20
I do not want to optimize the rotation ;) I need the Matrix in oder to plug it into an optimizer, otherwise it's quite inconvenient. And yes obviously the matrix would be sparse there should be quite exactly 4 to 9 entries in every row, like usual pde's, where people also generate the matrix. Shouldn't be too inefficient. — E.G.A.L., Sep 04 '21 at 22:19
"I do not want to optimize the rotation ;) I need the Matrix in order to plug it into an optimizer": mh, isn't that contradictory ? — , Sep 05 '21 at 08:54
No it's not. ;) I want to something like: min_x f(A@x). Where the solution Algorithm, needs the first and second derivative of f(A@x). I am also aware that the adjoint map of a rotation is the inverse rotation, since the operation is unitary/orthogonal. However then I could not use a preimplemented solver or it's getting quite inconvenient :D. And since the representation matrix is sparse I thought it should be easy to compute. :D — E.G.A.L., Sep 06 '21 at 20:14

score 1 · Answer 1 · answered Sep 01 '21 at 07:49

I like your approach but there is a slight misconception in it. What you want to transform are not the pixel values themselves but the coordinates. So you don't reshape your image but rather do a np.indices on it to obtain coordinates to each pixel. For those a rotation around a point looks like

rotation_matrix@(coordinates-fixed_point)+fixed_point

except that I have to transpose a bit to get the dimensions to align. The cove below is a slight adoption of my code in this answer.

As an example I am going to use the Wikipedia-logo-v2 by Nohat. It is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.

First I read in the picture, swap x and y axis to not get mad and rotate the coordinates as described above.

import numpy as np
import matplotlib.pyplot as plt
import itertools

image = plt.imread('wikipedia.jpg')
image = np.swapaxes(image,0,1)/255

fixed_point = np.array(image.shape[:2], dtype='float')/2
points = np.moveaxis(np.indices(image.shape[:2]),0,-1).reshape(-1,2)
a = 2*np.pi/8
A = np.array([[np.cos(a),-np.sin(a)],[np.sin(a),np.cos(a)]])
rotated_coordinates = (A@(points-fixed_point.reshape(1,2)).T).T+fixed_point.reshape(1,2)

Now I set up a little class to interpolate between the pixels that do not fit exactly to an other pixel. And finally I swap the axis back and plot it.

class Image_knn():
   def fit(self, image):
       self.image = image.astype('float')

   def predict(self, x, y):
       image = self.image
       weights_x = [(1-(x % 1)).reshape(*x.shape,1), (x % 1).reshape(*x.shape,1)]
       weights_y = [(1-(y % 1)).reshape(*x.shape,1), (y % 1).reshape(*x.shape,1)]
       start_x = np.floor(x)
       start_y = np.floor(y)
       return sum([image[np.clip(np.floor(start_x + x), 0, image.shape[0]-1).astype('int'),
                         np.clip(np.floor(start_y + y), 0, image.shape[1]-1).astype('int')] * weights_x[x]*weights_y[y] 
                   for x,y in itertools.product(range(2),range(2))])

   
image_model = Image_knn()
image_model.fit(image)

transformed_image = image_model.predict(*rotated_coordinates.T).reshape(*image.shape)
plt.imshow(np.swapaxes(transformed_image,0,1))

And I get a result like this

Possible Issue

The artifact in the bottom left that looks like one needs to clean the screen comes from the following problem: When we rotate it can happen that we don't have enough pixels to paint the lower left. What we do by default in image_knn is to clip the coordinates to an area where we have information. That means when we ask image knn for pixels coming from outside the image it gives us the pixels at the boundary of the image. This looks good if there is a background but if an object touches the edge of the picture it looks odd like here. Just something to keep in mind when using this.

score 0 · Answer 2 · answered Sep 02 '21 at 09:26

Thank you for your answer!

But actually it is not a misconception that you could let this roation be represented by a matrix multiplication with the reshaped vector. I used your code to generate such a matrix (its surely not the most efficient way but it works, most likely you see a more efficient implementation immediately XD. You see I really need it as a matix multiplication :-D). What I basically did is to generate the representation matrix of the linear transformation, by computing how every of the 100*100 basis images (i.e. the image with zeros everywhere und a one) is mapped by your transformation.

import sys
import numpy as np
import matplotlib.pyplot as plt
import itertools

angle = 2*np.pi/6
image_expl = plt.imread('wikipedia.jpg')
image_expl = image_expl[:,:,0]
plt.imshow(image_expl)
plt.title("Image")
plt.show()
image_shape = image_expl.shape
pixel_number = image_shape[0]*image_shape[1]
rot_mat = np.zeros((pixel_number,pixel_number))
for i in range(pixel_number):
    vector = np.zeros(pixel_number)
    vector[i] = 1
    image = vector.reshape(*image_shape)
    fixed_point = np.array(image.shape, dtype='float')/2
    points = np.moveaxis(np.indices(image.shape),0,-1).reshape(-1,2)
    a = -angle
    A = np.array([[np.cos(a),-np.sin(a)],[np.sin(a),np.cos(a)]])
    rotated_coordinates = (A@(points-fixed_point.reshape(1,2)).T).T+fixed_point.reshape(1,2)
    x,y = rotated_coordinates.T
    image = image.astype('float')
    weights_x = [(1-(x % 1)).reshape(*x.shape), (x % 1).reshape(*x.shape)]
    weights_y = [(1-(y % 1)).reshape(*x.shape), (y % 1).reshape(*x.shape)]
    start_x = np.floor(x)
    start_y = np.floor(y)
    transformed_image_returned = sum([image[np.clip(np.floor(start_x + x), 0, image.shape[0]-1).astype('int'),
                         np.clip(np.floor(start_y + y), 0, image.shape[1]-1).astype('int')] * weights_x[x]*weights_y[y] 
                   for x,y in itertools.product(range(2),range(2))])
    rot_mat[:,i] = transformed_image_returned
    if i%100 == 0: print(int(100*i/pixel_number), "% finisched")


plt.imshow((rot_mat @ image_expl.reshape(-1)).reshape(image_shape))

Thank you again :-)

Rotating a reshaped image as a matrix operation

2 Answers2

Possible Issue