22

I'm trying to figure out how to sample all of the pixels in an image and generate a palette of colors from it, something like this or this. I have no idea where to even begin. Can anyone point me in the right direction?

__EDIT: __

This is what I've ended up with so far:

I used this Pixelate function to get large block sections like joe_coolish suggested. It's working perfectly and giving me a pretty good sample of colors to work with (this is from the windows sample jelly fish picture):

Now, if someone could help me get the 5 most distinct colors (darkest blue, lightest blue, orange, gray and peach(?)), I would love you forever. I really don't understand how to average or add colors together. I also can't figure out how to tell if a color is similar programatically, there are some many numbers and variables in you explanations that I get lost trying to figure out what's doing what to whom.

Peter O.
  • 32,158
  • 14
  • 82
  • 96
scottm
  • 27,829
  • 22
  • 107
  • 159
  • 3
    Questions and answers like these make me proud to be part of this community. – Adriano Carneiro Apr 28 '11 at 20:16
  • 1
    I see you added more to the question. do you still need help adding and averaging colors? – joe_coolish May 15 '11 at 01:40
  • 1
    Finding the palette of colors that matches best a true-color image is an old problem that several algorithms try to solve. You can start with the Wikipedia page about [Color Quantization](http://en.wikipedia.org/wiki/Color_quantization), and check this [list of algorithms](https://github.com/jsummers/imageworsener/issues/2). – wip Dec 03 '13 at 07:50

6 Answers6

28

The answers involving the code show you how to get the full palette. If you want to get the average colors like in the websites you posted, this is how I would do it.

Source Image:

Source

First, I would average the colors by applying a lowpass filter (something like a Gaussian Blur)

enter image description here

That way you are limiting the total palette. From there I would divide the screen into N blocks (N being the total number of pixels you want in your palette)

enter image description here

From there, target each block and iterate over each pixel, and get the average pixel for that block and add it to your palette index. The result is something like this:

enter image description here

That way your palette is limited and you get the average colors from the different regions. You can do all of that in code, and if you'd like some help with that, let me know and I'll post some. This is just the High Level "what I would do".

joe_coolish
  • 7,201
  • 13
  • 64
  • 111
  • That definitely seems to make the most sense. I can do most of that, the part I don't know how to do is get the "average pixel" for a region. Just the one that occurs the most? – scottm Apr 28 '11 at 19:54
  • 1
    There are different techniques. Lets say you are working in 24x24 blocks and your image is 6x4 blocks (144x96 pixels). For block 8 (index 1,1) I would determine the pixels from the smoothed image and add all of the color values together and take the average. In this example it would be pixels 24-48 x 24-48. I would also avoid RGB and use HSI or HSL, and then average those three values. That will give you a better result – joe_coolish Apr 28 '11 at 20:06
  • Just out of interest, why is the blur step required? Could averaging each block work just as well? – Thomas Bratt Jun 25 '12 at 13:18
  • 1
    The lowpass filter (the blur) is used to smooth out the image and remove "artifacts". You could average all the colors in the cubes and be fine, but some times you get "artifacts" in the cube. E.g. a plane flying in the first cube would cause random reds, greens and blues to be averaged in with the sky. Blurring will smooth out these sharp color changes for a more predictable result. Either way is fine. – joe_coolish Jun 25 '12 at 14:06
  • 1
    This will inevitably remove highlights you'll want to keep though. There's a lot of cyan in there because there's a lot of sea area, but you don't really need all _that_ many tints of that. Meanwhile you lose all of the yellow beach colour. – Nyerguds Apr 24 '17 at 10:59
  • 2
    This doesn’t do a good job of finding prominent colors in the image. It finds averages of different prominent colors, almost guaranteeing test mind of the colors found are actually present in the image. Instead of white for the clouds and blue for the sky, you get lots of light blues and grays. – Cris Luengo May 18 '18 at 01:29
  • @CrisLuengo Yes, if I were to do this again today (this is a 7 year old answer. Gosh, has it been that long?) I would use some of the prominent Machine Learning algorithms to extract primary, secondary foreground and background colors. I wouldn't do this manually. ML has come ALONG way! – joe_coolish May 23 '18 at 15:39
  • 2
    I'm not even talking about machine learning. This problem had been solved when you posted this answer, several decades prior, actually. "Computer algorithms to perform color quantization on bitmaps have been studied since the 1970s." ([quoting Wikipedia](https://en.wikipedia.org/wiki/Color_quantization)). Sorry for being so negative about your solution, I'm just really surprised that this gets so many upvotes... :) – Cris Luengo May 23 '18 at 15:52
  • 1
    @CrisLuengo "I'm just really surprised that this gets so many upvotes... :)" Actually, me too! I was still in college when I wrote this and for a long time, this was my most upvoted comment. Like I said, I would do this very differently now. – joe_coolish Jun 05 '18 at 15:55
6

First, take the pixels in the picture: (assumes using System.Drawing.Imaging; and using System.Runtime.InteropServices)

Bitmap b = new Bitmap(myImage);
BitmapData bd = b.LockBits(new Rectangle(0, 0, b.Width, b.Height), ImageLockMode.ReadOnly, ImageFormat.Format32Bpp);
int[] arr = new int[bd.Width * bd.Height - 1];
Marshal.Copy(bd.Scan0, arr, 0, arr.Length);
b.UnlockBits(bd);

Then you can create your palette:

var distinctColors = arr.Distinct();

Optionally: Eliminate similar colors until you have your preferred palette size. Here's how you might do that (though this is most definitely not the most efficient or accurate way, just the simplest):

var dc = distinctColors.toArray(); // int dc[] = distinctColors.toArray() is what it used to be
int cmIndex1 = -1;
int cmIndex2 = -1;
int cmDiff = -1;
for (int i = 0; i < dc.length; i++) {
    Color c1 = Color.FromArgb(dc[i]);
    for (int j = i + 1; j < dc.length; j++) {
        Color c2 = Color.FromArgb(dc[j]);
        // Note: you might want to include alpha below
        int diff = Math.Abs(c1.R - c2.R) + Math.Abs(c1.G - c2.G) + Math.Abs(c1.B - c2.B);
        if (cmDiff < 0 || diff < cmDiff) {
            cmIndex1 = i;
            cmIndex2 = j;
            cmDiff = diff;
        }
    }
}
// Remove the colors, replace with average, repeat until you have the desired number of colors
Ry-
  • 218,210
  • 55
  • 464
  • 476
  • Is there a good method to eliminate similar colors? I knew I could just iterate pixels to get all of the distinct colors, but I want to try to eliminate the similar ones until there are only 4 or 5 left – scottm Apr 28 '11 at 19:34
  • 1
    Yes, there are many ways. I think the simplest would be: while there are more than 5 colors, loop through all colors, then inside that loop loop through all colors comparing the (A)RGB sum of the color to the color in the outer loop. Keep 3 variables for the closest matches (difference, match 1, match 2). Remove the two closest and replace with the color average. Sorry if that's a bit confusing :P – Ry- Apr 28 '11 at 19:37
5

It is likely in any rich image that most of your colors will be unique in some way. It would follow, then, that fetching distinct colors will likely not help you accomplish your goal.

I recommend inspecting the HSV values for each pixel in your image. I'll leave you to countless online examples of retrieving images as arrays of HSV values.

With your HSV values, you can calculate clusters of prominent hues by create an integer array of 256 hue counts, computing a histogram of hues in your image data. You can determine prominent hues by finding clusters of 4-6 sequential hues with a high count sum.

After picking several prominent hues, subdivide pixels of those hues into another histogram measuring saturation, and pick out prominent clusters, and so on.

Rough Example

The code below makes some attempt to help identify prominent hues. There are most likely other awesome ways to do this; however, this may provide some ideas.

First, I get all the image colors as an array of Color objects, like so:

private static Color[] GetImageData(Image image)
{
    using (var b = new Bitmap(image))
    {
        var bd = b.LockBits(new Rectangle(0, 0, b.Width, b.Height), ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
        byte[] arr = new byte[bd.Width * bd.Height * 3];
        Color[] colors = new Color[bd.Width * bd.Height];
        Marshal.Copy(bd.Scan0, arr, 0, arr.Length);
        b.UnlockBits(bd);

        for (int i = 0; i < colors.Length; i++)
        {
            var start = i*3;
            colors[i] = Color.FromArgb(arr[start], arr[start + 1], arr[start + 2]);
        }

        return colors;
    }
}

You might consider validating that I got the order of RGB in the Color.FromArgb method call in the correct order.

Next, I stash aside a utility method for converting to HSV. In my example, I'll only work with hues, but here's a full working example of the conversion:

private static void ColorToHSV(Color color, out int hue, out int saturation, out int value)
{
    int max = Math.Max(color.R, Math.Max(color.G, color.B));
    int min = Math.Min(color.R, Math.Min(color.G, color.B));

    hue = (int)(color.GetHue() * 256f / 360f);
    saturation = (max == 0) ? 0 : (int)(1d - (1d * min / max));
    value = (int)(max / 255d);
}

Finally, I build the hue histogram, define a width of hues (say, 9 hues) in which to aggregate counts together, and then I report the counts to the console.

private static void ProcessImage(Color[] imagecolors)
{
    var hues = new int[256];
    var hueclusters = new int[256];
    int hue, saturation, value;

    // build hue histogram.
    foreach (var color in imagecolors) {
        ColorToHSV(color, out hue, out saturation, out value);
        hues[hue]++;
    }

    // calculate counts for clusters of colors.
    for (int i = 0; i < 256; i++) {
        int huecluster = 0;
        for (int count = 0, j = i; count < 9; count++, j++) {
            huecluster += hues[j % 256];
        }

        hueclusters[(i + 5) % 256] = huecluster;
    }

    // Print clusters on the console
    for (int i = 0; i < 256; i++) {
        Console.WriteLine("Hue {0}, Score {1}.", i, hueclusters[i]);
    }
}

I've made no attempt to filter down to which hues to choose. You may need to consider some heuristics rather than blindly picking the top so-many counts, because you probably want to pick hues that are somewhat separated on the color spectrum. I haven't time to explore this any further, but I hope this provides some insight into a strategy you can consider.

kbrimington
  • 25,142
  • 5
  • 62
  • 74
  • That should be `new byte[bd.Height * bd.Stride]` - [lines always align to 4 bytes](https://stackoverflow.com/q/2185944/395685), so on 3 bytes per colour, the line length will rarely match `width * 3` exactly. This also means you need to specifically iterate over y and x to keep these padding bytes into account. – Nyerguds May 17 '18 at 18:51
4

I started here:

System.Drawing.Image img = System.Drawing.Bitmap.FromFile("file");
System.Drawing.Imaging.ColorPalette palette = img.Palette;
foreach (Color color in palette.Entries)
{
  //...
}
Gary Kindel
  • 17,071
  • 7
  • 49
  • 66
  • This will only work with bitmaps which have 256 colours or less, otherwise the `img.Palette.Entries` property won't contain any entries (the pixels map to an RGB colour value instead of a colour palette) – Chris Oct 15 '13 at 08:46
  • This whole question is irrelevant if the image already has a palette, so I'm fairly sure it doesn't... – Nyerguds Jan 10 '18 at 13:40
3

I'm going to describe the best approach at a very high level.

First you construct a histogram of colors in the picture and their frequency.

You end up with a list of all colors in the image, you can use data clustering to find candidate colors to merge. Colors that are merged together into a weighted average based on the frequency of the original colors.

This way you can incrementally decrease the palette to a desired fidelity, while preserving high contrast but fine details and only losing fidelity where the gradients are much more subtle.

Once you have the reduced palette, you recolor the picture using the nearest neighbor color that is in the palette.

  • 1
    Interesting that it took 7 years for someone to find this question filled with amateur answers and finally post the correct, established solution. – Cris Luengo May 18 '18 at 01:34
  • 1
    See [this answer to a different question](https://stackoverflow.com/a/49715101/7328782) for a short description of some algorithms actually in use in the industry for this purpose. – Cris Luengo May 18 '18 at 01:38
  • Excellent. Thank you very much, Cris – SteamyThePunk May 18 '18 at 23:00
2

The K-Means clustering algorithm works well for this problem. It does a great job of extracting centroids of image color clusters, but be aware that its non-deterministic behavior makes determining the actual prominence of each cluster difficult.

snort
  • 2,285
  • 3
  • 19
  • 21