1

For a school project, our informatics teacher wants us to reinvent the wheel. We have given an array, representing the pixels of an image, containing Colour-Objects defined in another script. They represent a set of 4 Integers, with the Values 0 to 255 for Red, Green, Blue and Alpha Values. Now we have to do the standard operations for image manipulation on this array. We were explicitly told, to use the Internet and question-site's like stack-overflow for reference.

For which I have no approach: How to convert a given Colour-Object-Array to another Array representing the same Image, but rotated by x degrees(with expansion). Where do the new Colours/Pixels land, and how to calculate that? How to compute the new size of this Array? Is there any easy-to-understand pdf, I could work through, to understand, how, f.e. the PIL image.rotate(expand=true) algorithm works in theory, or could anybody come up with an explanation how to do this? I would appreciate pseudo-code or python 3, due to it's the only programming-language I understand.

Short example for such an Array:

BLUE  = Colour(0  ,0  ,255,255)
BLACK = Colour(0  ,0  ,0  ,255)
WHITE = Colour(255,255,255,255)
Array = [ [BLUE , BLACK, WHITE, BLUE ],
          [BLACK, BLACK, BLUE , WHITE],
          [WHITE, WHITE, BLUE , WHITE] ]

Edit: To access the Colour-Values, there are the methods getred(), getgreen(), getblue() and gettuple() - I have already implemented the "painters"-algorithm, meaning Colours can be merged by calling merge(bottomColour, topColour) this returns the resulting colour, if one is placed ontop of the other. Theory of this is found here: Determine RGBA colour received by combining two colours

We are not allowed to use numpy, or any other modules or libraries. Places where no Colour/Pixel is, should be 'None'.

Big Thanks in advance!

Blarify
  • 29
  • 1
  • 11
  • are you supposed to rotate the image to any angle or is it just 90 degree steps so you can stick to the limited set of named colors? – Piglet Jun 26 '17 at 13:49
  • any angle that could be achieved with integers, I have already googled a lot, if it was only 90 degrees, I wouldn't come up with this question :D – Blarify Jun 26 '17 at 13:52
  • Even if you are allowed by your teacher to ask here, you should follow the rules: First do the work yourself, as much as you can, and after that ask us about questions and problems that appeared. Don't forget to explain how they appeared. As it is, it is a forbidden format of questions. – Gangnus Jun 26 '17 at 14:01
  • but how are you supposed to present the results? you have blue, black and white. you'll end up with colors you don't have a name for. are you supposed to give sets of numbers instead? – Piglet Jun 26 '17 at 14:02
  • @piglet I have made an edit, for how to access the ColourObject, I didn't knew rotating will result in new Colours. – Blarify Jun 26 '17 at 14:10
  • @Gangnus Actually I did all I could do, of the task of our teacher gave us, with my actual knowledge, the reason why I didn't posted it, was because it's irrelevant to the question, nobody want's to read useless information for a lifetime long. I have no Idea about anything with rotation, that's the reason why I'm asking for the theory. :) – Blarify Jun 26 '17 at 14:13
  • Sorry, so, it is already the filtered task, only the problem part remained? Then it is OK, but it is hard to undrestand that from the post text. My answer is below. – Gangnus Jun 26 '17 at 14:16
  • if you rotate an image your pixels "contents/colors" will end up at new coordiantes which are not necessarily integer coordinates. so you need some heuristic to resample the image. just google image rotation and read a few links. – Piglet Jun 26 '17 at 14:24
  • use nearest neighbor (will result in the same colors) simply loop through all pixels of target array , compute corresponding position in source array (by rotating loops `x,y` back and round to integer) and copy cell value from source to target if rotated position is inside array if not use some background color like black. see [Rotating a square TBitmap on its center](https://stackoverflow.com/a/44299929/2521214) – Spektre Jun 27 '17 at 07:36

2 Answers2

4

We need to map each coordinate in the rotated image to a corresponding coordinate in the original.

Assuming a rotation about (a, b) and counter-clockwise rotation by θ degrees:

enter image description here

Where (x, y) are in the original image and (x', y') the rotated one.


Simple technique: nearest neighbor

When sampling the pixel data using the calculated coordinates, we could simply round them to the nearest integers (i.e. nearest pixel). This gives the following result:

enter image description here

At first glance this seems good enough, but webpage re-scaling + image compression blurs the edges. A zoomed-in view reveals that the resulting image has nasty jagged edges (aliasing):

enter image description here


Filtering: bilinear approximation

To improve on this, we need to realize that the rotated "pixel" area actually covers multiple pixels in the original image:

enter image description here

We can then calculate the average pixel color as the sum of contributions from each covered original pixel weighted by their relative areas. Let's call this "anisotropic" filtering for convenience (not the exact meaning of this term, but the closest I can think of).

However the areas will be quite difficult to calculate exactly. So we can "cheat" a little by applying an approximation, where the rotated sample area (in red) is aligned with the gridlines:

enter image description here

This makes the areas a lot easier to calculate. We shall use the first-order linear average method - "bilinear" filtering.

C# code sample:

Transform trn = new Transform(a, cx, cy); // inverse rotation transform to original image space

for (int y = 0; y < h; y++)
{
    for (int x = 0; x < w; x++)
    {
        Vector v = trn.Get((float)x, (float)y);
        int i = (int)Math.Floor(v.x),
            j = (int)Math.Floor(v.y);
        float s = v.x - (float)i,
              t = v.y - (float)j;
                
        RGB c = RGB.Black, u; float z, r = 0.0f;
        if ((u = src.getPixel(i, j)).Valid)
        {
            z = (1 - s) * (1 - t); // area of overlap in top-left covered pixel
            c += u * z; r += z; // add to total color and total area
        }
        if ((u = src.getPixel(i + 1, j)).Valid)
        {
            z = s * (1 - t);
            c += u * z; r += z;
        }
        if ((u = src.getPixel(i, j + 1)).Valid)
        {
            z = (1 - s) * t;
            c += u * z; r += z;
        }
        if ((u = src.getPixel(i + 1, j + 1)).Valid)
        {
            z = s * t;
            c += u * z; r += z;
        }
        
        if (r > 0.0f)
            dst.setPixel(x, y, c * (1.0f / r)); // normalize the sum by total area
    }
}

Zoomed-in result:

enter image description here

Much better than the naive nearest-neighbor method!


OCD Alert!

Just out of curiosity, I implemented the full "anisotropic" method mentioned before. Took way longer than it should, and not exactly efficient (using Sutherland-Hodgman clipping to calculate the intersection region between the rotated pixel area and each grid pixel). Computational time went through the roof - about 7 seconds compared to less than 0.5 for the bilinear method. The end result? Not worth the effort at all!

(L: bilinear, R: anisotropic)

sdsds

Code (my implementation is trash, don't bother to read it, really):

private static Vector[][] clipboxes = new Vector[][] {
    new Vector[] { new Vector(-1f,-1f), new Vector(0f,-1f), new Vector(0f,0f), new Vector(-1f,0f)},
    new Vector[] { new Vector(0f,-1f), new Vector(1f,-1f), new Vector(1f,0f), new Vector(0f,0f)},
    new Vector[] { new Vector(1f,-1f), new Vector(2f,-1f), new Vector(2f,0f), new Vector(1f,0f)},
    new Vector[] { new Vector(-1f,0f), new Vector(0f,0f), new Vector(0f,1f), new Vector(-1f,1f)},
    new Vector[] { new Vector(0f,0f), new Vector(1f,0f), new Vector(1f,1f), new Vector(0f,1f)},
    new Vector[] { new Vector(1f,0f), new Vector(2f,0f), new Vector(2f,1f), new Vector(1f,1f)},
    new Vector[] { new Vector(-1f,1f), new Vector(0f,1f), new Vector(0f,2f), new Vector(-1f,2f)},
    new Vector[] { new Vector(0f,1f), new Vector(1f,1f), new Vector(1f,2f), new Vector(0f,2f)},
    new Vector[] { new Vector(1f,1f), new Vector(2f,1f), new Vector(2f,2f), new Vector(1f,2f)}
};

private static bool inside(Vector a, Vector b, Vector c)
{
    return ((c - b) ^ (a - b)) > 0f;
}

private static Vector intersect(Vector a, Vector b, Vector c, Vector d)
{
    return (((c - d) * (a ^ b)) - ((a - b) * (c ^ d))) * (1.0f / ((a - b) ^ (c - d)));
}

private static float getArea(List<Vector> l)
{
    if (l.Count == 0) 
        return 0f;
    float sum = 0.0f;
    Vector b = l.Last();
    foreach (Vector c in l)
    {
        sum += b ^ c;
        b = c;
    }
    return 0.5f * Math.Abs(sum);
}

private static float getOverlap(Vector[] clip, Vector[] box)
{
    List<Vector> lO = box.ToList();
    Vector lC = clip[clip.Length - 1];
    foreach (Vector C in clip)
    {   
        if (lO.Count == 0)
            return 0.0f;
        List<Vector> lI = lO;
        Vector lB = lI.Last();
        lO = new List<Vector>();
        foreach (Vector B in lI)
        {
            if (inside(B, lC, C))
            {
                if (!inside(lB, lC, C))
                    lO.Add(intersect(lB, B, lC, C));
                lO.Add(B);
            }
            else
            if (inside(lB, lC, C)) 
                lO.Add(intersect(lB, B, lC, C));
            lB = B;
        }
        lC = C;
    }
    return getArea(lO);
}

// image processing code, as before

    Transform trn = new Transform(a, cx, cy);

    for (int y = 0; y < h; y++)
    {
        for (int x = 0; x < w; x++)
        {
            Vector p = trn.Get((float)x, (float)y);
            int i = p.X, j = p.Y;
            Vector d = new Vector(i, j);
            
            List<Vector> r = new List<Vector>();
            r.Add(p - d);
            r.Add(trn.Get((float)(x+1), (float)y) - d);
            r.Add(trn.Get((float)(x+1), (float)(y+1)) - d);
            r.Add(trn.Get((float)x, (float)(y+1)) - d);
            
            RGB c = RGB.Black;
            float t = 0.0f;
            
            for (int l = 0; l < 3; l++)
            {
                for (int m = 0; m < 3; m++)
                {
                    float area = getOverlap(clipboxes[m * 3 + l], r.ToArray());
                    if (area > 0.0f)
                    {
                        RGB s = src.getPixel(i + l - 1, j + m - 1);
                        if (s.Valid)
                        {
                            c += s * area;
                            t += area;
                        }
                    }
                }
            }
            
            if (t > 0.0f)
                dst.setPixel(x, y, c * (1.0f / t));
        }
    }

There are more advanced techniques available, e.g. using Fourier transforms - see Dartmouth University's paper named High Quality Alias Free Image Rotation (directly available from their website). Also, instead of bilinear interpolation, we could have used a higher order e.g. bicubic, which would give even smoother results.

meowgoesthedog
  • 14,670
  • 4
  • 27
  • 40
  • I assume, the secound formula to be "y=..." instead of "x=...", am I right? – Blarify Jun 28 '17 at 22:59
  • @PythonNewbie sorry yes, fixed that for you – meowgoesthedog Jun 29 '17 at 07:51
  • 1
    perfect, thanks, that two formulas where exactly, what I was looking for :D! – Blarify Jun 29 '17 at 18:49
  • Great answer! Unfortunately, as of 16/MAY/2021, the URL http://www.cs.dartmouth.edu/reports/TR96-301.pdf is not working. Could you substitute it with an exact name of a paper? There are at least several opapers that I found using "Fourier transforms inurl:dartmouth.edu". I think others would find it useful, too. – Curious Watcher May 16 '21 at 16:16
  • 1
    @AleksandrasUrbonas the paper is called **High Quality Alias Free Image Rotation**; fortunately I was able to recover it with the wayback machine, and it's still directly available from the university's website – meowgoesthedog May 17 '21 at 09:40
  • @meowgoesthedog this is great! please update your answer and we can close my question. You rock! – Curious Watcher May 17 '21 at 13:07
-1

As for placement, try to make the back transformation. Let every result pixel square be represented by its central point. For every such point let's find where belong the source point. Find, to what pixel square belongs that source point and - voila - you have the source square and source set of colours.

Notice, that using the straight order, from source to result, if you want to get the picture without holes, or overlapping, you have to find the areas of the result transformed pixels. It is very complicated. On the other hand, the reversed task can be done fast and simple.

Don't forget to count sine and cosine once and use them for every point calculation.

Gangnus
  • 24,044
  • 16
  • 90
  • 149
  • @Piglet The question is: where the pixel land. I say, it is better to reverse the task and to look for source for every land point. – Gangnus Jun 26 '17 at 14:12
  • That means, iterating through the resulting array and compute with a formula, from which element/elements of the starting array to get the data? Could you link me maybe a wikipedia-article or something like that to this formula, if it exists and does it have a name? Due to I'm only an 11th class pupil, I'm not that experienced in maths, as that I could come up with such a formula myself.(I have only basic knownledge about sine and cosine, because that's a topic of the 12th class.) – Blarify Jun 26 '17 at 14:31
  • @Gangnus I think you've skipped a couple of steps that the OP would need to know. It is correct that they need to perform the inverse transformation, but without knowing what the forward transformation is, it's not very helpful. Also, I do not understand your last sentence. What do you mean by "count"? – beaker Jun 26 '17 at 14:35
  • @beaker 1. counting = counting the transformation. 2. The transformation formulae are given in every book or article on plane turn. – Gangnus Jun 26 '17 at 14:48
  • @PythonNewbie I had invented that way for myself about 15 years ago. It is so trivial that I had never searched if it was used already. I am sure it was. As for formulae, look in https://en.wikipedia.org/wiki/Transformation_matrix for Examples in 2D computer graphics. Of course, to do back transformation, you have to use - angle instead of angle and 1/stretch instead of stretch. Really, you should find info in wiki yourself. – Gangnus Jun 26 '17 at 14:52
  • 1
    @Gangnus 1. After pondering about this for a while, I think you might mean "calculate". "Count" does not have the same meaning. 2. Of course the formulae are widely available, and of course they're simple, but that is what the question is asking for. If you didn't want to answer the question, then why did you post an answer? – beaker Jun 26 '17 at 15:07
  • Agree with `beaker`: dear @Gangnus, please share some specifics, since the answers reads like "go figure" - which is not helpful. I wish you could share a piece of your code from way back - it must have been a novel thing 15 years ago.. Either way, totally up to you :) Right now, voted answer down. – Curious Watcher May 17 '21 at 13:10
  • 1
    @AleksandrasUrbonas The answers to algorithm questions should not be language-specific. So, no code. If you do not understand the details of the algorithm, ask. If you cannot make code in the programming language you practise, up to the English text explaining the algorithm, ask questions on the language or look for ready answers. If I see that the text explanation is too long and complicated, I use pseudocode. But this text is short, and if you cannot make translation English->specific language, the double translation pseudocode->English->specific language is totally impossible for you. – Gangnus May 17 '21 at 14:50