79

My application: I am trying to rotate an image (using OpenCV and Python)

Rotating Images

At the moment I have developed the below code which rotates an input image, padding it with black borders, giving me A. What I want is B - the largest possible area crop window within the rotated image. I call this the axis-aligned boundED box.

This is essentially the same as Rotate and crop, however I cannot get the answer on that question to work. Additionally, that answer is apparently only valid for square images. My images are rectangular.

Code to give A:

import cv2
import numpy as np


def getTranslationMatrix2d(dx, dy):
    """
    Returns a numpy affine transformation matrix for a 2D translation of
    (dx, dy)
    """
    return np.matrix([[1, 0, dx], [0, 1, dy], [0, 0, 1]])


def rotateImage(image, angle):
    """
    Rotates the given image about it's centre
    """

    image_size = (image.shape[1], image.shape[0])
    image_center = tuple(np.array(image_size) / 2)

    rot_mat = np.vstack([cv2.getRotationMatrix2D(image_center, angle, 1.0), [0, 0, 1]])
    trans_mat = np.identity(3)

    w2 = image_size[0] * 0.5
    h2 = image_size[1] * 0.5

    rot_mat_notranslate = np.matrix(rot_mat[0:2, 0:2])

    tl = (np.array([-w2, h2]) * rot_mat_notranslate).A[0]
    tr = (np.array([w2, h2]) * rot_mat_notranslate).A[0]
    bl = (np.array([-w2, -h2]) * rot_mat_notranslate).A[0]
    br = (np.array([w2, -h2]) * rot_mat_notranslate).A[0]

    x_coords = [pt[0] for pt in [tl, tr, bl, br]]
    x_pos = [x for x in x_coords if x > 0]
    x_neg = [x for x in x_coords if x < 0]

    y_coords = [pt[1] for pt in [tl, tr, bl, br]]
    y_pos = [y for y in y_coords if y > 0]
    y_neg = [y for y in y_coords if y < 0]

    right_bound = max(x_pos)
    left_bound = min(x_neg)
    top_bound = max(y_pos)
    bot_bound = min(y_neg)

    new_w = int(abs(right_bound - left_bound))
    new_h = int(abs(top_bound - bot_bound))
    new_image_size = (new_w, new_h)

    new_midx = new_w * 0.5
    new_midy = new_h * 0.5

    dx = int(new_midx - w2)
    dy = int(new_midy - h2)

    trans_mat = getTranslationMatrix2d(dx, dy)
    affine_mat = (np.matrix(trans_mat) * np.matrix(rot_mat))[0:2, :]
    result = cv2.warpAffine(image, affine_mat, new_image_size, flags=cv2.INTER_LINEAR)

    return result
Community
  • 1
  • 1
aaronsnoswell
  • 6,051
  • 5
  • 47
  • 69
  • 1
    As far as I can see, this is essentially a non-linear optimisation problem, (search all AABB rectangles contained in the rotated image to find the one with the largest area). I just can't seem to figure out the logic required to solve it. – aaronsnoswell May 22 '13 at 23:04
  • 3
    Here's a link to the algorithm of someone who worked on the same problem. http://roffle-largest-rectangle.blogspot.com/2011/09/find-largest-rectangle-in-rotated-image.html It's Java code and I haven't checked its logic but it may help you to get started. – Alexey May 22 '13 at 23:46
  • 2
    Also this http://stackoverflow.com/questions/5789239/calculate-largest-rectangle-in-a-rotated-rectangle – Alexey May 22 '13 at 23:58
  • Hey! Absolutely brilliant - both those links look perfect. – aaronsnoswell May 23 '13 at 00:09
  • Glad to help - that's an interesting math / geometry problem. – Alexey May 23 '13 at 00:22
  • I just went and do the maths... while you guys Googling for existing answer. Anyway, if none of that works just call me. I wrote down the term for cut-off area to be minimized and just find where the derivative of that equals to zero. That's the approach. – Apiwat Chantawibul May 23 '13 at 00:29
  • From the above mentioned links only this solution is correct: http://stackoverflow.com/questions/5789239/calculate-largest-rectangle-in-a-rotated-rectangle#7519376 – coproc May 24 '13 at 13:39
  • http://stackoverflow.com/a/22511805/2106820 – Arfan Mirza Mar 19 '14 at 16:12
  • 2
    As a side note: Step A can be done with much less code, see here: http://www.pyimagesearch.com/2017/01/02/rotate-images-correctly-with-opencv-and-python/ (implemented in the function [`rotate_bound()` of imutils](https://github.com/jrosebr1/imutils/blob/master/imutils/convenience.py#L41) – luator Apr 06 '17 at 08:08
  • Looking for a solution to a related problem: given the target height and width (smaller than the original height and width), what should the angle of rotation be? Appreciate any pointers! – Lakshay Sharma Oct 01 '20 at 05:28
  • Thanks, I also found very useful your code about how to correctly rotate an image. – Francesco Pagani Aug 24 '21 at 09:45

12 Answers12

116

The math behind this solution/implementation is equivalent to this solution of an analagous question, but the formulas are simplified and avoid singularities. This is python code with the same interface as largest_rotated_rect from the other solution, but giving a bigger area in almost all cases (always the proven optimum):

def rotatedRectWithMaxArea(w, h, angle):
  """
  Given a rectangle of size wxh that has been rotated by 'angle' (in
  radians), computes the width and height of the largest possible
  axis-aligned rectangle (maximal area) within the rotated rectangle.
  """
  if w <= 0 or h <= 0:
    return 0,0

  width_is_longer = w >= h
  side_long, side_short = (w,h) if width_is_longer else (h,w)

  # since the solutions for angle, -angle and 180-angle are all the same,
  # if suffices to look at the first quadrant and the absolute values of sin,cos:
  sin_a, cos_a = abs(math.sin(angle)), abs(math.cos(angle))
  if side_short <= 2.*sin_a*cos_a*side_long or abs(sin_a-cos_a) < 1e-10:
    # half constrained case: two crop corners touch the longer side,
    #   the other two corners are on the mid-line parallel to the longer line
    x = 0.5*side_short
    wr,hr = (x/sin_a,x/cos_a) if width_is_longer else (x/cos_a,x/sin_a)
  else:
    # fully constrained case: crop touches all 4 sides
    cos_2a = cos_a*cos_a - sin_a*sin_a
    wr,hr = (w*cos_a - h*sin_a)/cos_2a, (h*cos_a - w*sin_a)/cos_2a

  return wr,hr

Here is a comparison of the function with the other solution:

>>> wl,hl = largest_rotated_rect(1500,500,math.radians(20))
>>> print (wl,hl),', area=',wl*hl
(828.2888697391496, 230.61639227890998) , area= 191016.990904
>>> wm,hm = rotatedRectWithMaxArea(1500,500,math.radians(20))
>>> print (wm,hm),', area=',wm*hm
(730.9511000407718, 266.044443118978) , area= 194465.478358

With angle angle in [0,pi/2[ the bounding box of the rotated image (width w, height h) has these dimensions:

  • width w_bb = w*cos_a + h*sin_a
  • height h_bb = w*sin_a + h*cos_a

If w_r, h_r are the computed optimal width and height of the cropped image, then the insets from the bounding box are:

  • in horizontal direction: (w_bb-w_r)/2
  • in vertical direction: (h_bb-h_r)/2

Proof:

Looking for the axis aligned rectangle between two parallel lines that has maximal area is an optimization problem with one parameter, e.g. x as in this figure: animated parameter

Let s denote the distance between the two parallel lines (it will turn out to be the shorter side of the rotated rectangle). Then the sides a, b of the sought-after rectangle have a constant ratio with x, s-x, resp., namely x = a sin α and (s-x) = b cos α:

enter image description here

So maximizing the area a*b means maximizing x*(s-x). Because of "theorem of height" for right-angled triangles we know x*(s-x) = p*q = h*h. Hence the maximal area is reached at x = s-x = s/2, i.e. the two corners E, G between the parallel lines are on the mid-line:

enter image description here

This solution is only valid if this maximal rectangle fits into the rotated rectangle. Therefore the diagonal EG must not be longer than the other side l of the rotated rectangle. Since

EG = AF + DH = s/2*(cot α + tan α) = s/(2sin αcos α) = s/sin 2*α

we have the condition s ≤ lsin 2α, where s and l are the shorter and longer side of the rotated rectangle.

In case of s > lsin 2α the parameter x must be smaller (than s/2) and s.t. all corners of the sought-after rectangle are each on a side of the rotated rectangle. This leads to the equation

x*cot α + (s-x)*tan α = l

giving x = sin α*(lcos α - ssin α)/cos 2*α. From a = x/sin α and b = (s-x)/cos α we get the above used formulas.

coproc
  • 6,027
  • 2
  • 20
  • 31
  • 3
    +1 - I've tested your solution against an optimization procedure (maximize the area) and your solution, always faster and more precise, has given the same results so far... – Saullo G. P. Castro May 29 '13 at 20:13
  • @Saullo math can sometimes do magic! :-) thank you for sharing your findings. I have now added a (mostly graphical) derivation of the formulas. – coproc May 31 '13 at 12:09
  • thank you! your answer is the best one and helped me a lot! – khamitimur Sep 16 '16 at 13:25
  • 1
    So how do I get the coordinates of this box? – Toke Faurby Dec 27 '17 at 22:30
  • 1
    @TokeFaurby good question! I have added the distances of the borders of the cropped area from the bounding box (just before the **Proof** section) – coproc Dec 28 '17 at 19:46
  • hint for anybody else confused: the arguments `w` and `h` to rotatedRectWithMaxArea should correspond to the sizes of the first and second axes of the image array, nothing to do with which side is actually shorter/longer. – Yibo Yang Apr 09 '18 at 05:18
  • I would like to cite this proof on a paper. If this is your proof, did you publish it somewhere? If not, where can I find the original author? @coproc – Gorayni Apr 24 '18 at 17:21
  • @Gorayni yes, this proof is my approach. It is not published anywhere else. Just go ahead and use it (reformulate it to your needs) - it is rather straight forward anyway. – coproc May 02 '18 at 08:55
  • I made a java port while working in similar problem in android https://gist.github.com/omkar-tenkale/3da22fd414737e6f1f55024367004fbc – Omkar T Sep 20 '20 at 16:07
35

So, after investigating many claimed solutions, I have finally found a method that works; The answer by Andri and Magnus Hoff on Calculate largest rectangle in a rotated rectangle.

The below Python code contains the method of interest - largest_rotated_rect - and a short demo.

import math
import cv2
import numpy as np


def rotate_image(image, angle):
    """
    Rotates an OpenCV 2 / NumPy image about it's centre by the given angle
    (in degrees). The returned image will be large enough to hold the entire
    new image, with a black background
    """

    # Get the image size
    # No that's not an error - NumPy stores image matricies backwards
    image_size = (image.shape[1], image.shape[0])
    image_center = tuple(np.array(image_size) / 2)

    # Convert the OpenCV 3x2 rotation matrix to 3x3
    rot_mat = np.vstack(
        [cv2.getRotationMatrix2D(image_center, angle, 1.0), [0, 0, 1]]
    )

    rot_mat_notranslate = np.matrix(rot_mat[0:2, 0:2])

    # Shorthand for below calcs
    image_w2 = image_size[0] * 0.5
    image_h2 = image_size[1] * 0.5

    # Obtain the rotated coordinates of the image corners
    rotated_coords = [
        (np.array([-image_w2,  image_h2]) * rot_mat_notranslate).A[0],
        (np.array([ image_w2,  image_h2]) * rot_mat_notranslate).A[0],
        (np.array([-image_w2, -image_h2]) * rot_mat_notranslate).A[0],
        (np.array([ image_w2, -image_h2]) * rot_mat_notranslate).A[0]
    ]

    # Find the size of the new image
    x_coords = [pt[0] for pt in rotated_coords]
    x_pos = [x for x in x_coords if x > 0]
    x_neg = [x for x in x_coords if x < 0]

    y_coords = [pt[1] for pt in rotated_coords]
    y_pos = [y for y in y_coords if y > 0]
    y_neg = [y for y in y_coords if y < 0]

    right_bound = max(x_pos)
    left_bound = min(x_neg)
    top_bound = max(y_pos)
    bot_bound = min(y_neg)

    new_w = int(abs(right_bound - left_bound))
    new_h = int(abs(top_bound - bot_bound))

    # We require a translation matrix to keep the image centred
    trans_mat = np.matrix([
        [1, 0, int(new_w * 0.5 - image_w2)],
        [0, 1, int(new_h * 0.5 - image_h2)],
        [0, 0, 1]
    ])

    # Compute the tranform for the combined rotation and translation
    affine_mat = (np.matrix(trans_mat) * np.matrix(rot_mat))[0:2, :]

    # Apply the transform
    result = cv2.warpAffine(
        image,
        affine_mat,
        (new_w, new_h),
        flags=cv2.INTER_LINEAR
    )

    return result


def largest_rotated_rect(w, h, angle):
    """
    Given a rectangle of size wxh that has been rotated by 'angle' (in
    radians), computes the width and height of the largest possible
    axis-aligned rectangle within the rotated rectangle.

    Original JS code by 'Andri' and Magnus Hoff from Stack Overflow

    Converted to Python by Aaron Snoswell
    """

    quadrant = int(math.floor(angle / (math.pi / 2))) & 3
    sign_alpha = angle if ((quadrant & 1) == 0) else math.pi - angle
    alpha = (sign_alpha % math.pi + math.pi) % math.pi

    bb_w = w * math.cos(alpha) + h * math.sin(alpha)
    bb_h = w * math.sin(alpha) + h * math.cos(alpha)

    gamma = math.atan2(bb_w, bb_w) if (w < h) else math.atan2(bb_w, bb_w)

    delta = math.pi - alpha - gamma

    length = h if (w < h) else w

    d = length * math.cos(alpha)
    a = d * math.sin(alpha) / math.sin(delta)

    y = a * math.cos(gamma)
    x = y * math.tan(gamma)

    return (
        bb_w - 2 * x,
        bb_h - 2 * y
    )


def crop_around_center(image, width, height):
    """
    Given a NumPy / OpenCV 2 image, crops it to the given width and height,
    around it's centre point
    """

    image_size = (image.shape[1], image.shape[0])
    image_center = (int(image_size[0] * 0.5), int(image_size[1] * 0.5))

    if(width > image_size[0]):
        width = image_size[0]

    if(height > image_size[1]):
        height = image_size[1]

    x1 = int(image_center[0] - width * 0.5)
    x2 = int(image_center[0] + width * 0.5)
    y1 = int(image_center[1] - height * 0.5)
    y2 = int(image_center[1] + height * 0.5)

    return image[y1:y2, x1:x2]


def demo():
    """
    Demos the largest_rotated_rect function
    """

    image = cv2.imread("lenna_rectangle.png")
    image_height, image_width = image.shape[0:2]

    cv2.imshow("Original Image", image)

    print "Press [enter] to begin the demo"
    print "Press [q] or Escape to quit"

    key = cv2.waitKey(0)
    if key == ord("q") or key == 27:
        exit()

    for i in np.arange(0, 360, 0.5):
        image_orig = np.copy(image)
        image_rotated = rotate_image(image, i)
        image_rotated_cropped = crop_around_center(
            image_rotated,
            *largest_rotated_rect(
                image_width,
                image_height,
                math.radians(i)
            )
        )

        key = cv2.waitKey(2)
        if(key == ord("q") or key == 27):
            exit()

        cv2.imshow("Original Image", image_orig)
        cv2.imshow("Rotated Image", image_rotated)
        cv2.imshow("Cropped Image", image_rotated_cropped)

    print "Done"


if __name__ == "__main__":
    demo()

Image Rotation Demo

Simply place this image (cropped to demonstrate that it works with non-square images) in the same directory as the above file, then run it.

Community
  • 1
  • 1
aaronsnoswell
  • 6,051
  • 5
  • 47
  • 69
  • 4
    The function `largest_rotated_rect` gives rectangle dimensions that cannot be extended, i.e. no axis parallel rectangle bigger in both dimensions will fit into the rotated rectangle. But except for a few special cases this function will not return the largest (maximal area) rectangle dimensions fitting in. See my solution for the true optimum. – coproc May 27 '13 at 18:29
  • You can use `cv::RotatedRect(center,ImageSize,angle).boundingRect()` to find the size of the rotated image – user362515 Feb 13 '14 at 18:47
  • rotate_image actually takes the angle in degrees, not radians, since cv2.getRotationMatrix2D takes the angle in degrees, not radians http://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html#getrotationmatrix2d – wordsforthewise Jun 14 '16 at 05:01
  • I think you have a typo. Your results for `gamma` in the `largest_rotated_rect` method will always be the same. – enobrev Aug 22 '16 at 10:03
  • 1
    Hi. Can you tell me how get rotated image with white background not black? – Tegos Mar 18 '17 at 19:55
  • For choice of background colour add `borderValue` parameter to `cv2.warpAffine` method. For white, use `borderValue=(255, 255, 255)` – MukeshD Aug 07 '21 at 12:50
15

Congratulations for the great work! I wanted to use your code in OpenCV with the C++ library, so I did the conversion that follows. Maybe this approach could be helpful to other people.

#include <iostream>
#include <opencv.hpp>

#define PI 3.14159265359

using namespace std;

double degree_to_radian(double angle)
{
    return angle * PI / 180;
}

cv::Mat rotate_image (cv::Mat image, double angle)
{
    // Rotates an OpenCV 2 image about its centre by the given angle
    // (in radians). The returned image will be large enough to hold the entire
    // new image, with a black background

    cv::Size image_size = cv::Size(image.rows, image.cols);
    cv::Point image_center = cv::Point(image_size.height/2, image_size.width/2);

    // Convert the OpenCV 3x2 matrix to 3x3
    cv::Mat rot_mat = cv::getRotationMatrix2D(image_center, angle, 1.0);
    double row[3] = {0.0, 0.0, 1.0};
    cv::Mat new_row = cv::Mat(1, 3, rot_mat.type(), row);
    rot_mat.push_back(new_row);


    double slice_mat[2][2] = {
        {rot_mat.col(0).at<double>(0), rot_mat.col(1).at<double>(0)},
        {rot_mat.col(0).at<double>(1), rot_mat.col(1).at<double>(1)}
    };

    cv::Mat rot_mat_nontranslate = cv::Mat(2, 2, rot_mat.type(), slice_mat);

    double image_w2 = image_size.width * 0.5;
    double image_h2 = image_size.height * 0.5;

    // Obtain the rotated coordinates of the image corners
    std::vector<cv::Mat> rotated_coords;

    double image_dim_d_1[2] = { -image_h2, image_w2 };
    cv::Mat image_dim = cv::Mat(1, 2, rot_mat.type(), image_dim_d_1);
    rotated_coords.push_back(cv::Mat(image_dim * rot_mat_nontranslate));


    double image_dim_d_2[2] = { image_h2, image_w2 };
    image_dim = cv::Mat(1, 2, rot_mat.type(), image_dim_d_2);
    rotated_coords.push_back(cv::Mat(image_dim * rot_mat_nontranslate));


    double image_dim_d_3[2] = { -image_h2, -image_w2 };
    image_dim = cv::Mat(1, 2, rot_mat.type(), image_dim_d_3);
    rotated_coords.push_back(cv::Mat(image_dim * rot_mat_nontranslate));


    double image_dim_d_4[2] = { image_h2, -image_w2 };
    image_dim = cv::Mat(1, 2, rot_mat.type(), image_dim_d_4);
    rotated_coords.push_back(cv::Mat(image_dim * rot_mat_nontranslate));


    // Find the size of the new image
    vector<double> x_coords, x_pos, x_neg;
    for (int i = 0; i < rotated_coords.size(); i++)
    {
        double pt = rotated_coords[i].col(0).at<double>(0);
        x_coords.push_back(pt);
        if (pt > 0)
            x_pos.push_back(pt);
        else
            x_neg.push_back(pt);
    }

    vector<double> y_coords, y_pos, y_neg;
    for (int i = 0; i < rotated_coords.size(); i++)
    {
        double pt = rotated_coords[i].col(1).at<double>(0);
        y_coords.push_back(pt);
        if (pt > 0)
            y_pos.push_back(pt);
        else
            y_neg.push_back(pt);
    }


    double right_bound = *max_element(x_pos.begin(), x_pos.end());
    double left_bound = *min_element(x_neg.begin(), x_neg.end());
    double top_bound = *max_element(y_pos.begin(), y_pos.end());
    double bottom_bound = *min_element(y_neg.begin(), y_neg.end());

    int new_w = int(abs(right_bound - left_bound));
    int new_h = int(abs(top_bound - bottom_bound));

    // We require a translation matrix to keep the image centred
    double trans_mat[3][3] = {
        {1, 0, int(new_w * 0.5 - image_w2)},
        {0, 1, int(new_h * 0.5 - image_h2)},
        {0, 0, 1},
    };


    // Compute the transform for the combined rotation and translation
    cv::Mat aux_affine_mat = (cv::Mat(3, 3, rot_mat.type(), trans_mat) * rot_mat);
    cv::Mat affine_mat = cv::Mat(2, 3, rot_mat.type(), NULL);
    affine_mat.push_back(aux_affine_mat.row(0));
    affine_mat.push_back(aux_affine_mat.row(1));

    // Apply the transform
    cv::Mat output;
    cv::warpAffine(image, output, affine_mat, cv::Size(new_h, new_w), cv::INTER_LINEAR);

    return output;
}

cv::Size largest_rotated_rect(int h, int w, double angle)
{
    // Given a rectangle of size wxh that has been rotated by 'angle' (in
    // radians), computes the width and height of the largest possible
    // axis-aligned rectangle within the rotated rectangle.

    // Original JS code by 'Andri' and Magnus Hoff from Stack Overflow

    // Converted to Python by Aaron Snoswell (https://stackoverflow.com/questions/16702966/rotate-image-and-crop-out-black-borders)
    // Converted to C++ by Eliezer Bernart

    int quadrant = int(floor(angle/(PI/2))) & 3;
    double sign_alpha = ((quadrant & 1) == 0) ? angle : PI - angle;
    double alpha = fmod((fmod(sign_alpha, PI) + PI), PI);

    double bb_w = w * cos(alpha) + h * sin(alpha);
    double bb_h = w * sin(alpha) + h * cos(alpha);

    double gamma = w < h ? atan2(bb_w, bb_w) : atan2(bb_h, bb_h);

    double delta = PI - alpha - gamma;

    int length = w < h ? h : w;

    double d = length * cos(alpha);
    double a = d * sin(alpha) / sin(delta);
    double y = a * cos(gamma);
    double x = y * tan(gamma);

    return cv::Size(bb_w - 2 * x, bb_h - 2 * y);
}

// for those interested in the actual optimum - contributed by coproc
#include <algorithm>
cv::Size really_largest_rotated_rect(int h, int w, double angle)
{
  // Given a rectangle of size wxh that has been rotated by 'angle' (in
  // radians), computes the width and height of the largest possible
  // axis-aligned rectangle within the rotated rectangle.
  if (w <= 0 || h <= 0)
    return cv::Size(0,0);

  bool width_is_longer = w >= h;
  int side_long = w, side_short = h;
  if (!width_is_longer)
    std::swap(side_long, side_short);

  // since the solutions for angle, -angle and pi-angle are all the same,
  // it suffices to look at the first quadrant and the absolute values of sin,cos:
  double sin_a = fabs(sin(angle)), cos_a = fabs(cos(angle));
  double wr,hr;
  if (side_short <= 2.*sin_a*cos_a*side_long)
  {
    // half constrained case: two crop corners touch the longer side,
    // the other two corners are on the mid-line parallel to the longer line
    double x = 0.5*side_short;
    wr = x/sin_a;
    hr = x/cos_a;
    if (!width_is_longer)
      std::swap(wr,hr);
  }
  else
  { 
    // fully constrained case: crop touches all 4 sides
    double cos_2a = cos_a*cos_a - sin_a*sin_a;
    wr = (w*cos_a - h*sin_a)/cos_2a;
    hr = (h*cos_a - w*sin_a)/cos_2a;
  }

  return cv::Size(wr,hr);
}

cv::Mat crop_around_center(cv::Mat image, int height, int width)
{
    // Given a OpenCV 2 image, crops it to the given width and height,
    // around it's centre point

    cv::Size image_size = cv::Size(image.rows, image.cols);
    cv::Point image_center = cv::Point(int(image_size.height * 0.5), int(image_size.width * 0.5));

    if (width > image_size.width)
        width = image_size.width;

    if (height > image_size.height)
        height = image_size.height;

    int x1 = int(image_center.x - width  * 0.5);
    int x2 = int(image_center.x + width  * 0.5);
    int y1 = int(image_center.y - height * 0.5);
    int y2 = int(image_center.y + height * 0.5);


    return image(cv::Rect(cv::Point(y1, x1), cv::Point(y2,x2)));
}

void demo(cv::Mat image)
{
    // Demos the largest_rotated_rect function
    int image_height = image.rows;
    int image_width = image.cols;

    for (float i = 0.0; i < 360.0; i+=0.5)
    {
        cv::Mat image_orig = image.clone();
        cv::Mat image_rotated = rotate_image(image, i);

        cv::Size largest_rect = largest_rotated_rect(image_height, image_width, degree_to_radian(i));
        // for those who trust math (added by coproc):
        cv::Size largest_rect2 = really_largest_rotated_rect(image_height, image_width, degree_to_radian(i));
        cout << "area1 = " << largest_rect.height * largest_rect.width << endl;
        cout << "area2 = " << largest_rect2.height * largest_rect2.width << endl;

        cv::Mat image_rotated_cropped = crop_around_center(
                    image_rotated,
                    largest_rect.height,
                    largest_rect.width
                    );

        cv::imshow("Original Image", image_orig);
        cv::imshow("Rotated Image", image_rotated);
        cv::imshow("Cropped image", image_rotated_cropped);

        if (char(cv::waitKey(15)) == 'q')
            break;
    }

}

int main (int argc, char* argv[])
{
    cv::Mat image = cv::imread(argv[1]);

    if (image.empty())
    {
        cout << "> The input image was not found." << endl;
        exit(EXIT_FAILURE);
    }

    cout << "Press [s] to begin or restart the demo" << endl;
    cout << "Press [q] to quit" << endl;

    while (true)
    {
        cv::imshow("Original Image", image);
        char opt = char(cv::waitKey(0));
        switch (opt) {
        case 's':
            demo(image);
            break;
        case 'q':
            return EXIT_SUCCESS;
        default:
            break;
        }
    }

    return EXIT_SUCCESS;
}
kmkaplan
  • 18,655
  • 4
  • 51
  • 65
Eliezer Bernart
  • 2,386
  • 24
  • 33
  • 4
    I'm going to give you a 50 rep bounty at the end of a week. Thank you very very much for translating the code, dude. Fantastic! – karlphillip Nov 25 '14 at 22:54
7

Rotation and cropping in TensorFlow

I personally needed this function in TensorFlow and thanks for Aaron Snoswell, I could implement this function.

def _rotate_and_crop(image, output_height, output_width, rotation_degree, do_crop):
    """Rotate the given image with the given rotation degree and crop for the black edges if necessary
    Args:
        image: A `Tensor` representing an image of arbitrary size.
        output_height: The height of the image after preprocessing.
        output_width: The width of the image after preprocessing.
        rotation_degree: The degree of rotation on the image.
        do_crop: Do cropping if it is True.
    Returns:
        A rotated image.
    """

    # Rotate the given image with the given rotation degree
    if rotation_degree != 0:
        image = tf.contrib.image.rotate(image, math.radians(rotation_degree), interpolation='BILINEAR')

        # Center crop to ommit black noise on the edges
        if do_crop == True:
            lrr_width, lrr_height = _largest_rotated_rect(output_height, output_width, math.radians(rotation_degree))
            resized_image = tf.image.central_crop(image, float(lrr_height)/output_height)    
            image = tf.image.resize_images(resized_image, [output_height, output_width], method=tf.image.ResizeMethod.BILINEAR, align_corners=False)

    return image

def _largest_rotated_rect(w, h, angle):
    """
    Given a rectangle of size wxh that has been rotated by 'angle' (in
    radians), computes the width and height of the largest possible
    axis-aligned rectangle within the rotated rectangle.
    Original JS code by 'Andri' and Magnus Hoff from Stack Overflow
    Converted to Python by Aaron Snoswell
    Source: http://stackoverflow.com/questions/16702966/rotate-image-and-crop-out-black-borders
    """

    quadrant = int(math.floor(angle / (math.pi / 2))) & 3
    sign_alpha = angle if ((quadrant & 1) == 0) else math.pi - angle
    alpha = (sign_alpha % math.pi + math.pi) % math.pi

    bb_w = w * math.cos(alpha) + h * math.sin(alpha)
    bb_h = w * math.sin(alpha) + h * math.cos(alpha)

    gamma = math.atan2(bb_w, bb_w) if (w < h) else math.atan2(bb_w, bb_w)

    delta = math.pi - alpha - gamma

    length = h if (w < h) else w

    d = length * math.cos(alpha)
    a = d * math.sin(alpha) / math.sin(delta)

    y = a * math.cos(gamma)
    x = y * math.tan(gamma)

    return (
        bb_w - 2 * x,
        bb_h - 2 * y
    )

If you need further implementation of example and visualization in TensorFlow, you can use this repository. I hope this could be helpful to other people.

ByungSoo Ko
  • 503
  • 1
  • 6
  • 7
5

Inspired by Coprox's amazing work I wrote a function that forms together with Coprox's code a complete solution (so it can be used by copying & pasting with no-brainer). The rotate_max_area function below simply returns a rotated image without black boundary.

def rotate_bound(image, angle):
    # CREDIT: https://www.pyimagesearch.com/2017/01/02/rotate-images-correctly-with-opencv-and-python/
    (h, w) = image.shape[:2]
    (cX, cY) = (w // 2, h // 2)
    M = cv2.getRotationMatrix2D((cX, cY), -angle, 1.0)
    cos = np.abs(M[0, 0])
    sin = np.abs(M[0, 1])
    nW = int((h * sin) + (w * cos))
    nH = int((h * cos) + (w * sin))
    M[0, 2] += (nW / 2) - cX
    M[1, 2] += (nH / 2) - cY
    return cv2.warpAffine(image, M, (nW, nH))


def rotate_max_area(image, angle):
    """ image: cv2 image matrix object
        angle: in degree
    """
    wr, hr = rotatedRectWithMaxArea(image.shape[1], image.shape[0],
                                    math.radians(angle))
    rotated = rotate_bound(image, angle)
    h, w, _ = rotated.shape
    y1 = h//2 - int(hr/2)
    y2 = y1 + int(hr)
    x1 = w//2 - int(wr/2)
    x2 = x1 + int(wr)
    return rotated[y1:y2, x1:x2]
Naofumi
  • 331
  • 4
  • 9
5

A small update for brevity that makes use of the excellent imutils library.

def rotated_rect(w, h, angle):
    """
    Given a rectangle of size wxh that has been rotated by 'angle' (in
    radians), computes the width and height of the largest possible
    axis-aligned rectangle within the rotated rectangle.

    Original JS code by 'Andri' and Magnus Hoff from Stack Overflow

    Converted to Python by Aaron Snoswell
    """
    angle = math.radians(angle)
    quadrant = int(math.floor(angle / (math.pi / 2))) & 3
    sign_alpha = angle if ((quadrant & 1) == 0) else math.pi - angle
    alpha = (sign_alpha % math.pi + math.pi) % math.pi

    bb_w = w * math.cos(alpha) + h * math.sin(alpha)
    bb_h = w * math.sin(alpha) + h * math.cos(alpha)

    gamma = math.atan2(bb_w, bb_w) if (w < h) else math.atan2(bb_w, bb_w)

    delta = math.pi - alpha - gamma

    length = h if (w < h) else w

    d = length * math.cos(alpha)
    a = d * math.sin(alpha) / math.sin(delta)

    y = a * math.cos(gamma)
    x = y * math.tan(gamma)

    return (bb_w - 2 * x, bb_h - 2 * y)

def crop(img, w, h):
    x, y = int(img.shape[1] * .5), int(img.shape[0] * .5)

    return img[
        int(np.ceil(y - h * .5)) : int(np.floor(y + h * .5)),
        int(np.ceil(x - w * .5)) : int(np.floor(x + h * .5))
    ]

def rotate(img, angle):
    # rotate, crop and return original size
    (h, w) = img.shape[:2]
    img = imutils.rotate_bound(img, angle)
    img = crop(img, *rotated_rect(w, h, angle))
    img = cv2.resize(img,(w,h),interpolation=cv2.INTER_AREA)
    return img
  • Nice piece of code. I would like to create a new solution using your code as benchmark (since it is the lastest version of the algorithm) but I cannot find exactly where it is. Can you please point me out in which file of your github project is included this function? Thanks in advance. – Pablo Gonzalez Nov 14 '18 at 06:57
2

Swift solution

Thanks to coproc for his great solution. Here is the code in swift

// Given a rectangle of size.width x size.height that has been rotated by 'angle' (in
// radians), computes the width and height of the largest possible
// axis-aligned rectangle (maximal area) within the rotated rectangle.
func rotatedRectWithMaxArea(size: CGSize, angle: CGFloat) -> CGSize {
    let w = size.width
    let h = size.height

    if(w <= 0 || h <= 0) {
        return CGSize.zero
    }

    let widthIsLonger = w >= h
    let (sideLong, sideShort) = widthIsLonger ? (w, h) : (w, h)

    // since the solutions for angle, -angle and 180-angle are all the same,
    // if suffices to look at the first quadrant and the absolute values of sin,cos:
    let (sinA, cosA) = (sin(angle), cos(angle))
    if(sideShort <= 2*sinA*cosA*sideLong || abs(sinA-cosA) < 1e-10) {
        // half constrained case: two crop corners touch the longer side,
        // the other two corners are on the mid-line parallel to the longer line
        let x = 0.5*sideShort
        let (wr, hr) = widthIsLonger ? (x/sinA, x/cosA) : (x/cosA, x/sinA)
        return CGSize(width: wr, height: hr)
    } else {
        // fully constrained case: crop touches all 4 sides
        let cos2A = cosA*cosA - sinA*sinA
        let (wr, hr) = ((w*cosA - h*sinA)/cos2A, (h*cosA - w*sinA)/cos2A)
        return CGSize(width: wr, height: hr)
    }
}
Josh Bernfeld
  • 4,246
  • 2
  • 32
  • 35
2

Perhaps an even simplier solution would be:

def crop_image(image, angle):
    h, w = image.shape
    tan_a = abs(np.tan(angle * np.pi / 180))
    b = int(tan_a / (1 - tan_a ** 2) * (h - w * tan_a))
    d = int(tan_a / (1 - tan_a ** 2) * (w - h * tan_a))
    return image[d:h - d, b:w - b]

Instead of calculating the height and width of the rotated rectangle like many have done, it is sufficient to find the height of the black triangles that form when rotating an image.

Facorazza
  • 317
  • 1
  • 15
1

Correction to the most favored solution above given by Coprox on May 27 2013: when cosa = cosb infinity results in the last two lines. Solve by adding "or cosa equal cosb" in the preceding if selector.

Addition: if you do not know the original non-rotated nx and ny but only have the rotated frame (or image) then find the box just containing this (I do this by removing blank = monochrome borders) and first run the program reversely on its size to find nx and ny. If the image was rotated into a too small frame so that it was cut along the sides (into octagonal shape) I first find the x and y extensions to the full containment frame. However, this also does not work for angles around 45 degrees where the result gets square instead of maintaining the non-rotated aspect ratio. For me this routine only works properly up to 30 degrees.

Still a great routine! It solved my nagging problem in astronomical image alignment.

Rob Rutten
  • 111
  • 4
  • You mean the case `sin(a) = cos(a)`? Then indeed `cos(2a)` would be zero (because of `a = pi/4`), which is a singularity of the else-branch. With exact calculations we would never get into the else-branch, because `2*sin(a)*cos(a)` equals `1` for `a = pi/4` and `side_short <= side_long` holds by definition. But because of rounding errors the `if`-condition could still be false for `side_short ~= side_long` and `a ~= pi/4`. So I have extended the condition by `or abs(sin_a - cos_a) < 1e-10` to stay away from that singularity. Thank you for your hint! – coproc Dec 28 '17 at 14:44
0

Rotate images in correct order

 import cv2
 import pytesseract
 import urllib
 import numpy as np
 import re
 import imutils #added
 import PIL

 image = cv2.imread('my_pdf_madan_m/page_1.jpg')

 gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
 gray = cv2.bitwise_not(gray)

 rot_data = pytesseract.image_to_osd(image);
 print("[OSD] "+rot_data)
 rot = re.search('(?<=Rotate: )\d+', 
 rot_data).group(0)

 angle = float(rot)

 # rotate the image to deskew it
 rotated = imutils.rotate_bound(image, angle) #added


 #  TODO: Rotated image can be saved here
 print(pytesseract.image_to_osd(rotated));

 # Run tesseract OCR on image
 text = pytesseract.image_to_string(rotated, 
 lang='eng', config="--psm 6")
 print(text)
0

Recently implemented a solution for Pytorch. It might come in handy. Could potentially be used with the 'Random Rotation Transform' as well. Just need to read the particular angle used by the transform and then just use it with PyTorch transforms. Function simply takes in a batch of images and does the random rotation with cropping.

import torchvision.transforms as transforms
import math

def _largest_rotated_rect(w, h, angle):
"""
Given a rectangle of size wxh that has been rotated by 'angle' (in
radians), computes the width and height of the largest possible
axis-aligned rectangle within the rotated rectangle.
Original JS code by 'Andri' and Magnus Hoff from Stack Overflow
Converted to Python by Aaron Snoswell
Source: http://stackoverflow.com/questions/16702966/rotate-image-and-crop-out-black-borders
"""

quadrant = int(math.floor(angle / (math.pi / 2))) & 3
sign_alpha = angle if ((quadrant & 1) == 0) else math.pi - angle
alpha = (sign_alpha % math.pi + math.pi) % math.pi

bb_w = w * math.cos(alpha) + h * math.sin(alpha)
bb_h = w * math.sin(alpha) + h * math.cos(alpha)

gamma = math.atan2(bb_w, bb_w) if (w < h) else math.atan2(bb_w, bb_w)

delta = math.pi - alpha - gamma

length = h if (w < h) else w

d = length * math.cos(alpha)
a = d * math.sin(alpha) / math.sin(delta)

y = a * math.cos(gamma)
x = y * math.tan(gamma)

return (
    bb_w - 2 * x,
    bb_h - 2 * y
)


def _rotate_and_crop(image, output_height=32, output_width=32):
"""Rotate the given image with the given rotation degree and crop for the black edges if necessary. For my case, image sizes are 32x32.
Args:
    image: A Batch of Tensors- normally from a dataloader.
    output_height: The height of the image after preprocessing.
    output_width: The width of the image after preprocessing.
Returns:
    A rotated image.
"""

# Rotate the given image with the given rotation degree
rotation_transform = transforms.RandomRotation((0, 360))
angle_rot = rotation_transform.angle_rot #you will have to read it from the pytorch library

lrr_width, lrr_height = _largest_rotated_rect(output_height, output_width, math.radians(angle_rot))
croped_image = transforms.CenterCrop((lrr_height, lrr_width))
resize_transform = transforms.Resize(size=(output_height, output_width))

transform = transforms.Compose([rotation_transform, croped_image, resize_transform, transforms.RandomHorizontalFlip(),
                                               transforms.ToTensor(),
                                               ])

image = transform(image)

return image
jmm
  • 1
  • 2
0

By doing the calculations by hand and looking at the original post, I found a minor typo on the gamma calculations. It should actually be:

gamma = math.atan2(bb_w, bb_h) if (w < h) else math.atan2(bb_h, bb_w)

eatsfood
  • 950
  • 2
  • 21
  • 31