JavaScript - Get average color from a certain area of an image

Question

I need to get the average color from a rectangle area of an image using JavaScript.

I tried using tracking.js but it doesn't allow to specify areas instead of single pixels.

Answer 1

If you need to get the average color of a single pixel, rather than the color of a rectangular area, please take a look at this other question:

Get pixel color from canvas, on mouseover

As you say that you need to get the color from a rectangle area from an image, I will assume you mean that you need to get the average color of a given area, and not the color of a single pixel.

Anyway, both are done in a very similar way:

Getting The Color/Value of A Single Pixel from An Image or Canvas

To get the color of a single pixel, you would first draw that image to a canvas:

const image = document.getElementById('image');
const canvas = document.createElement('canvas');
const context = canvas.getContext('2d');
const width = image.width;
const height = image.height;

canvas.width = width;
canvas.height = height;

context.drawImage(image, 0, 0, width, height);

And then get the value of a single pixel like this:

const data = context.getImageData(X, Y, 1, 1).data;

// RED   = data[0]
// GREEN = data[1]
// BLUE  = data[2]
// ALPHA = data[3]

✂️ Getting The Average Color/Value from A Region On An Image or Canvas

You need to use this same CanvasRenderingContext2D.getImageData() to get the values of a wider (multi-pixel) area, which you do by changing its third and fourth params. The signature of that function is:

ImageData ctx.getImageData(sx, sy, sw, sh);

• sx: The x coordinate of the upper left corner of the rectangle from which the ImageData will be extracted.
• sy: The y coordinate of the upper left corner of the rectangle from which the ImageData will be extracted.
• sw: The width of the rectangle from which the ImageData will be extracted.
• sh: The height of the rectangle from which the ImageData will be extracted.

You can see it returns a ImageData object, whatever that is. The important part here is that that object has a .data property which contains all our pixel values.

However, note that .data property is a 1-dimension Uint8ClampedArray, which means that all the pixel's components have been flattened, so you are getting something that looks like this:

Let's say you have a 2x2 image like this:

 RED PIXEL |       GREEN PIXEL
BLUE PIXEL | TRANSPARENT PIXEL

Then, you will get them like this:

[ 255, 0, 0, 255,    0, 255, 0, 255,    0, 0, 255, 255,    0, 0, 0, 0          ]
|   RED PIXEL   |    GREEN PIXEL   |     BLUE PIXEL   |    TRANSPAERENT  PIXEL |
|   1ST PIXEL   |      2ND PIXEL   |      3RD PIXEL   |             4TH  PIXEL | 

✨ Let's See It in Action

const avgSolidColor = document.getElementById('avgSolidColor');
const avgAlphaColor = document.getElementById('avgAlphaColor');
const avgSolidWeighted = document.getElementById('avgSolidWeighted');

const avgSolidColorCode = document.getElementById('avgSolidColorCode');
const avgAlphaColorCode = document.getElementById('avgAlphaColorCode');
const avgSolidWeightedCOde = document.getElementById('avgSolidWeightedCode');

const brush = document.getElementById('brush');
const image = document.getElementById('image');
const canvas = document.createElement('canvas');
const context = canvas.getContext('2d');
const width = image.width;
const height = image.height;

const BRUSH_SIZE = brush.offsetWidth;
const BRUSH_CENTER = BRUSH_SIZE / 2;
const MIN_X = image.offsetLeft + 4;
const MAX_X = MIN_X + width - BRUSH_SIZE;
const MIN_Y = image.offsetTop + 4;
const MAX_Y = MIN_Y + height - BRUSH_SIZE;

canvas.width = width;
canvas.height = height;

context.drawImage(image, 0, 0, width, height);

function sampleColor(clientX, clientY) {
  const brushX = Math.max(Math.min(clientX - BRUSH_CENTER, MAX_X), MIN_X);
  const brushY = Math.max(Math.min(clientY - BRUSH_CENTER, MAX_Y), MIN_Y);

  const imageX = brushX - MIN_X;
  const imageY = brushY - MIN_Y;
 
  let R = 0;
  let G = 0;
  let B = 0;
  let A = 0;
  let wR = 0;
  let wG = 0;
  let wB = 0;
  let wTotal = 0;

  const data = context.getImageData(imageX, imageY, BRUSH_SIZE, BRUSH_SIZE).data;
  
  const components = data.length;
  
  for (let i = 0; i < components; i += 4) {
    // A single pixel (R, G, B, A) will take 4 positions in the array:
    const r = data[i];
    const g = data[i + 1];
    const b = data[i + 2];
    const a = data[i + 3];
    
    // Update components for solid color and alpha averages:
    R += r;
    G += g;
    B += b;
    A += a;
    
    // Update components for alpha-weighted average:
    const w = a / 255;
    wR += r * w;
    wG += g * w;
    wB += b * w;
    wTotal += w;
  }
  
  const pixelsPerChannel = components / 4;
  
 // The | operator is used here to perform an integer division:

  R = R / pixelsPerChannel | 0;
  G = G / pixelsPerChannel | 0;
  B = B / pixelsPerChannel | 0;
  wR = wR / wTotal | 0;
  wG = wG / wTotal | 0;
  wB = wB / wTotal | 0;

  // The alpha channel need to be in the [0, 1] range:

  A = A / pixelsPerChannel / 255;
  
  // Update UI:
  
  requestAnimationFrame(() => {
    brush.style.transform = `translate(${ brushX }px, ${ brushY }px)`;

    avgSolidColorCode.innerText = avgSolidColor.style.background
      = `rgb(${ R }, ${ G }, ${ B })`;

    avgAlphaColorCode.innerText = avgAlphaColor.style.background
      = `rgba(${ R }, ${ G }, ${ B }, ${ A.toFixed(2) })`;

    avgSolidWeightedCode.innerText = avgSolidWeighted.style.background
      = `rgb(${ wR }, ${ wG }, ${ wB })`;
  });
}

document.onmousemove = (e) => sampleColor(e.clientX, e.clientY);
  
sampleColor(MIN_X, MIN_Y);

body {
  margin: 0;
  height: 100vh;
  display: flex;
  flex-direction: row;
  align-items: center;
  justify-content: center;
  cursor: crosshair;
  font-family: monospace;
}

#image {
  border: 4px solid white;
  border-radius: 2px;
  box-shadow: 0 0 32px 0 rgba(0, 0, 0, .25);
  width: 150px;
  box-sizing: border-box;
}

#brush {
  position: absolute;
  top: 0;
  left: 0;
  pointer-events: none;
  width: 50px;
  height: 50px;
  background: magenta;
  mix-blend-mode: exclusion;
}

#samples {
  position: relative;
  list-style: none;
  padding: 0;
  width: 250px;
}

#samples::before {
  content: '';
  position: absolute;
  top: 0;
  left: 27px;
  width: 2px;
  height: 100%;
  background: black;
  border-radius: 1px;
}

#samples > li {
  position: relative;
  display: flex;
  flex-direction: column;
  justify-content: center;
  padding-left: 56px;
}

#samples > li + li {
  margin-top: 8px;
}

.sample {
  position: absolute;
  top: 50%;
  left: 16px;
  transform: translate(0, -50%);
  display: block;
  width: 24px;
  height: 24px;
  border-radius: 100%;
  box-shadow: 0 0 16px 4px rgba(0, 0, 0, .25);  
  margin-right: 8px;
}

.sampleLabel {
  font-weight: bold;
  margin-bottom: 8px;
}

.sampleCode {
  
}

<img id="image" src="data:image/gif;base64,R0lGODlhSwBLAPEAACMfIO0cJAAAAAAAACH/C0ltYWdlTWFnaWNrDWdhbW1hPTAuNDU0NTUAIf4jUmVzaXplZCBvbiBodHRwczovL2V6Z2lmLmNvbS9yZXNpemUAIfkEBQAAAgAsAAAAAEsASwAAAv+Uj6mb4A+QY7TaKxvch+MPKpC0eeUUptdomOzJqnLUvnFcl7J6Pzn9I+l2IdfII8DZiCnYsYdK4qRTptAZwQKRVK71CusOgx2nFRrlhMu+33o2NEalC6S9zQvfi3Mlnm9WxeQ396F2+HcQsMjYGEBRVbhy5yOp6OgIeVIHpEnZyYCZ6cklKBJX+Kgg2riqKoayOWl2+VrLmtDqBptIOjZ6K4qAeSrL8PcmHExsgMs2dpyIxPpKvdhM/YxaTMW2PGr9GP76BN3VHTMurh7eoU14jsc+P845Vn6OTb/P/I68iYOfwGv+JOmRNHBfsV5ujA1LqM4eKDoNvXyDqItTxYX/DC9irKBlIhkKGPtFw1JDiMeS7CqWqySPZcKGHH/JHGgIpb6bCl1O0LmT57yCOqoI5UcU0YKjPXmFjMm0ZQ4NIVdGBdZRi9WrjLxJNMY1Yr4dYeuNxWApl1ALHb+KDHrTV1owlriedJgSr4Cybu/9dFiWYAagsqAGVkkzaZTAuqD9ywKWMUG9dCO3u2zWpVzIhpW122utZlrHnTN+Bq2Mqrlnqh8CQ+0Mrq3Kc++q7eo6dlB3rLuh3abPVbbbI2mxBdhWdsZhid8cr0oy9F08q0k5FXSadiyL1mF5z51a8VsQOp3/LlodkBfzmzWf2bOrtfzr48k/1hupDaLa9rUbO+zlwndfaOCURAXRNaCBqBT2BncJakWfTzSYkmCEFr60RX0V8sKaHOltCBJ1tAAFYhHaVVbig3jxp0IBADs=" >

<div id="brush"></div>

<ul id="samples">
  <li>
    <span class="sample" id="avgSolidColor"></span>
    <div class="sampleLabel">avgSolidColor</div>
    <div class="sampleCode" id="avgSolidColorCode">rgb(-, -, -)</div>
  </li>
  <li>
    <span class="sample" id="avgAlphaColor"></span>
    <div class="sampleLabel">avgAlphaColor</div>
    <div class="sampleCode" id="avgAlphaColorCode">rgba(-, -, -, -)</div>
  </li>
  <li>
    <span class="sample" id="avgSolidWeighted"></span>
    <div class="sampleLabel">avgSolidWeighted</div>
    <div class="sampleCode" id="avgSolidWeightedCode">rgba(-, -, -, -)</div>
  </li>
</ul>

⚠️ Note I'm using a small data URI to avoid Cross-Origin issues if I include an external image or an answer that is larger than allowed if I try to use a longer data URI.

️ These average colors look weird, don't they? (@SMT's comment)

If you move the brush to the upper-left corner, you will see avgSolidColor is almost black. That's because most pixels in that area are totally transparent, so their value is exactly or quite close to 0, 0, 0, 255. That means that for each of those we process, R, G and B don't change or change very little, while pixelsPerChannel still takes them into account, so we end up dividing a small number (as we are adding 0 for most of them) by a large one (the total number of pixels in the brush), which gives us a value quite close to 0 (black).

For example, if we have two pixels, 0, 0, 0, 255 and 255, 0, 0, 0, by looking at them we might expect the average for the R channel to be 255 (as one of them is completely transparent). However, it will be (0 + 255) / 2 | 1 = 127. But don't worry, we will see how to do that next.

On the other hand, avgAlphaColor looks grey. Well, that's actually not true, it just looks grey because we are now using the alpha channel, which makes it semitransparent and allows us to see the background of the page, which in this case is white.

Alpha-weighted average (@SMT's comment's solution)

Then, what can we do to fix this? Well, it turns out we just need to use the alpha channel as the weight for our (now weighted) average:

That means that if a pixel is r, g, b, a, where a is in the interval [0, 255], we will update our variables like so:

const w = a / 255; // w is in the interval [0, 1]
wR += r * w;
wG += g * w; 
wB += b * w; 
wTotal += w;

Note how the more transparent a pixel is (w closer to 0), the less we care about its values in our calculations.

Answer 2

Not sure whether this technique is feasible in the DOM with canvases and ImageData, but back in the day with Flash and Actionscript, there was a great shortcut. In my use case, the regions in question were uniform: all regions were squares of the same size, e.g. I had a 1280x720 BitmapData (equivalent of ImageData) coming in from the webcam or a video, and I needed the average colors to make a grid of e.g. 16x9 regions representing the input image. What I did was this: draw the entire input image into a bitmap with size values in pixels equal to the number of region columns and rows. E.g. 1280x720px image drawn into a 16x9px bitmap. Then simply use native methods to get the color of exactly one pixel in the scaled-down image. This method was as precise as doing it manually, and much much faster, also the code was much simpler.