Organizing felt tip pens: optimizing the arrangement of items in a 2D grid by similarity of adjacent items, using JS [updated]

Question

UPD: the question has been updated with specifics and code, see below.

Warning: This question is about optimizing an arrangement of items in a matrix. It is not about comparing colors. Initially, I have decided that providing context about my problem would help. I now regret this decision because the result was the opposite: too much irrelevant talk about colors and almost nothing about actual algorithms.

---

I've got a box of 80 felt tip pens for my kid, and it annoys me so much that they are not sorted.

I used to play a game called Blendoku on Android where you need to do just that: arrange colors in such a way that they form gradients, with nearby colors being the most similar:

It is easy and fun to organize colors in intersecting lines like a crossword. But with these sketch markers, I've got a full-fledged 2D grid. What makes it even worse, colors are not extracted from a uniform gradient.

This makes me unable to sort felt tip pens by intuition. I need to do it algorithmically!

Here's what I've got:

• Solid knowledge of JavaScript
• A flat array of color values of all pens
• A function distance(color1, color2) that shows how similar a color pair is. It returns a float between 0 and 100 where 0 means that colors are identical.

All I'm lacking is an algorithm.

A factorial of 80 is a number with 118 digits, which rules out brute forcing.

There might be ways to make brute forcing feasible:

• fix the position of a few pens (e. g. in corners) to reduce the number of possible combinations;
• drop branches that contain at least one pair of very dissimilar neighbours;
• stop after finding first satisfactory arrangement.

But I'm still lacking an actual algorithm even for than, not to mention a non-brute-forcey one.

PS Homework:

• Sorting a matrix by similarity -- no answers.
• Algorithm for optimal 2D color palette arrangement -- very similar question, no answers.
• How to sort colors in two dimensions? -- more than 50% of cells already contain correctly organized colors; unfamiliar programming language; the actual sorting solution is not explained.
• Sort Colour / Color Values -- single flat array.

Update

Goal

Arrange a predefined set of 80 colors in a 8×10 grid in such a way that colors form nice gradients without tearing.

For reasons described below, there is no definitive solution to this question, possible solution are prone to imperfect result and subjectiveness. This is expected.

Note that I already have a function that compares two colors and tells how similar they are.

Color space is 3D

Human eye has three types of receptors to distinguish colors. Human color space is three-dimensional (trichromatic).

There are different models for describing colors and they all are three-dimensional: RGB, HSL, HSV, XYZ, LAB, CMY (note that "K" in CMYK is only required because colored ink is not fully opaque and expensive).

For example, this palette:

...uses polar coordinates with hue on the angle and saturation on the radius. Without the third dimension (lightness), this palete is missing all the bright and dark colors: white, black, all the greys (except 50% grey in the center), and tinted greys.

This palette is only a thin slice of the HSL/HSV color space:

It is impossible to lay out all colors on a 2D grid in a gradient without tearing in the gradient.

For example, here are all the 32-bit RGB colors, enumerated in lexicographic order into a 2D grid. You can see that the gradient has a lot of tearing:

Thus, my goal is to find an arbitrary, "good enough" arrangment where neighbors are more or less similar. I'd rather sacrifice a bit of similarity than have a few very similar clusters with tearing between them.

This question is about optimizing the grid in JavaScript, not about comparing colors!

I have already picked a function to determine the similarity of colors: Delta E 2000. This function is specifically designed to reflect the subjective human perception of color similarity. Here is a whitepaper describing how it works.

This question is about optimizing the arrangement of items in a 2D grid in such a way that the similarity of each pair of adjacent items (vertical and horizontal) is as low as it gets.

The word "optimizing" is used not in a sense of making an algorithm run faster. It is in a sense of Mathematical optimization:

In the simplest case, an optimization problem consists of maximizing or minimizing a real function by systematically choosing input values from within an allowed set and computing the value of the function.

In my case:

• "The function" here means running the DeltaE.getDeltaE00(color1, color2) function for all adjacent items, the output is a bunch of numbers (142 of them... I think) reflecting how dissimilar all the adjacent pairs are.
• "Maximizing or minimizing" — the goal is to minimize the output of "the function".
• "An input value" — is a specific arrangement of 80 predefined items in the 8×10 grid. There are a total of 80! input values, which makes the task impossible to brute force on a home computer.

Note that I don't have a clear definition for the minimization criteria of "the function". If we simply use the smallest sum of all numbers, then the winning result might be a case where the sum is the lowest, but a few adjacent item pairs are very dissimilar.

Thus, "the function" should maybe take into account not only the sum of all comparisons, but also ensure that no comparisons are way off.

Possible paths for solving the issue

From my previous bounty attempt on this question, I've learned the following paths:

• genetic algorithm
• optimizer/solver library
• manual sorting with a some algorithmic help
• something else?

The optimizer/solver library solution is what I initially was hoping for. But the mature libraries such as CPLEX and Gurobi are not in JS. There are some JS libraries but they are not well documented and have no newbie tutorials.

The genetic algorithm approach is very exciting. But it requires concieving algorithms of mutating and mating specimen (grid arrangements). Mutating seems trivial: simply swap adjacent items. But I have no idea about mating. And I have little understanding of the whole thing in general.

Manual sorting suggestions seem promising at the first glance, but fall short when looking into them in depth. They also assume using algorithms to solve certain steps without providing actual algorithms.

Code boilerplate and color samples

I have prepared a code boilerplate in JS: https://codepen.io/lolmaus/pen/oNxGmqz?editors=0010

Note: the code takes a while to run. To make working with it easier, do the following:

• Login/sign up for CodePen in order to be able to fork the boilerplate.
• Fork the boilerplate.
• Go to Settings/Behavior and make sure automatic update is disabled.
• Resize panes to maximize the JS pane and minimize other panes.
• Go to Change view/Debug mode to open the result in a separate tab. This enables console.log(). Also, if code execution freezes, you can kill the render tab without losing access the coding tab.
• After making changes to code, hit save in the code tab, then refresh the render tab and wait.
• In order to include JS libraries, go to Settings/JS. I use this CDN to link to code from GitHub: https://www.jsdelivr.com/?docs=gh

---

Source data:

const data = [
  {index: 1, id: "1", name: "Wine Red", rgb: "#A35A6E"},
  {index: 2, id: "3", name: "Rose Red", rgb: "#F3595F"},
  {index: 3, id: "4", name: "Vivid Red", rgb: "#F4565F"},
  // ...
];

Index is one-based numbering of colors, in the order they appear in the box, when sorted by id. It is unused in code.

Id is the number of the color from pen manufacturer. Since some numbers are in form of WG3, ids are strings.

---

Color class.

This class provides some abstractions to work with individual colors. It makes it easy to compare a given color with another color.

  index;
  id;
  name;
  rgbStr;
  collection;
  
  constructor({index, id, name, rgb}, collection) {
    this.index = index;
    this.id = id;
    this.name = name;
    this.rgbStr = rgb;
    this.collection = collection;
  }
  
  // Representation of RGB color stirng in a format consumable by the `rgb2lab` function
  @memoized
  get rgbArr() {
    return [
      parseInt(this.rgbStr.slice(1,3), 16),
      parseInt(this.rgbStr.slice(3,5), 16),
      parseInt(this.rgbStr.slice(5,7), 16)
    ];
  }
  
  // LAB value of the color in a format consumable by the DeltaE function
  @memoized
  get labObj() {
    const [L, A, B] = rgb2lab(this.rgbArr);
    return {L, A, B};
  }

  // object where distances from current color to all other colors are calculated
  // {id: {distance, color}}
  @memoized
  get distancesObj() {
    return this.collection.colors.reduce((result, color) => {
      if (color !== this) {      
        result[color.id] = {
          distance: this.compare(color),
          color,
        };
      }
      
      return result;
    }, {});
  }
    
  // array of distances from current color to all other colors
  // [{distance, color}]
  @memoized
  get distancesArr() {
    return Object.values(this.distancesObj);
  }
  
  // Number reprtesenting sum of distances from this color to all other colors
  @memoized
  get totalDistance() {
    return this.distancesArr.reduce((result, {distance}) => {      
      return result + distance;
    }, 0); 
  }

  // Accepts another color instance. Returns a number indicating distance between two numbers.
  // Lower number means more similarity.
  compare(color) {
    return DeltaE.getDeltaE00(this.labObj, color.labObj);
  }
}

---

Collection: a class to store all the colors and sort them.

class Collection {
  // Source data goes here. Do not mutate after setting in the constructor!
  data;
  
  constructor(data) {
    this.data = data;
  }
  
  // Instantiates all colors
  @memoized
  get colors() {
    const colors = [];

    data.forEach((datum) => {
      const color = new Color(datum, this);
      colors.push(color);
    });
  
    return colors;    
  }

  // Copy of the colors array, sorted by total distance
  @memoized
  get colorsSortedByTotalDistance() {
    return this.colors.slice().sort((a, b) => a.totalDistance - b.totalDistance);
  }

  // Copy of the colors array, arranged by similarity of adjacent items
  @memoized
  get colorsLinear() {
    // Create copy of colors array to manipualte with
    const colors = this.colors.slice();
    
    // Pick starting color
    const startingColor = colors.find((color) => color.id === "138");
    
    // Remove starting color
    const startingColorIndex = colors.indexOf(startingColor);
    colors.splice(startingColorIndex, 1);
    
    // Start populating ordered array
    const result = [startingColor];
    
    let i = 0;
    
    while (colors.length) {
      
      if (i >= 81) throw new Error('Too many iterations');

      const color = result[result.length - 1];
      colors.sort((a, b) => a.distancesObj[color.id].distance - b.distancesObj[color.id].distance);
      
      const nextColor = colors.shift();
      result.push(nextColor);
    }
    
    return result;
  }

  // Accepts name of a property containing a flat array of colors.
  // Renders those colors into HTML. CSS makes color wrap into 8 rows, with 10 colors in every row.
  render(propertyName) {
    const html =
      this[propertyName]
        .map((color) => {
          return `
          <div
            class="color"
            style="--color: ${color.rgbStr};"
            title="${color.name}\n${color.rgbStr}"
          >
            <span class="color-name">
              ${color.id}
            </span>
          </div>
          `;
        })
        .join("\n\n");
    
    document.querySelector('#box').innerHTML = html;
    document.querySelector('#title').innerHTML = propertyName;
  }
}

---

Usage:

const collection = new Collection(data);

console.log(collection);

collection.render("colorsLinear"); // Implement your own getter on Collection and use its name here

---

Sample output:

Answer 1

I managed to find a solution with objective value 1861.54 by stapling a couple ideas together.

1. Form unordered color clusters of size 8 by finding a min-cost matching and joining matched subclusters, repeated three times. We use d(C1, C2) = ∑_{c1 in C1} ∑_{c2 in C2} d(c1, c2) as the distance function for subclusters C1 and C2.

2. Find the optimal 2 × 5 arrangement of clusters according to the above distance function. This involves brute forcing 10! permutations (really 10!/4 if one exploits symmetry, which I didn't bother with).

3. Considering each cluster separately, find the optimal 4 × 2 arrangement by brute forcing 8! permutations. (More symmetry breaking possible, I didn't bother.)

4. Brute force the 4¹⁰ possible ways to flip the clusters. (Even more symmetry breaking possible, I didn't bother.)

5. Improve this arrangement with local search. I interleaved two kinds of rounds: a 2-opt round where each pair of positions is considered for a swap, and a large-neighborhood round where we choose a random maximal independent set and reassign optimally using the Hungarian method (this problem is easy when none of the things we're trying to move can be next to each other).

The output looks like this:

Python implementation at https://github.com/eisenstatdavid/felt-tip-pens

Answer 2

The trick for this is to stop thinking about it as an array for a moment and anchor yourself to the corners.

First, you need to define what problem you are trying to solve. Normal colors have three dimensions: hue, saturation, and value (darkness), so you're not going to be able to consider all three dimensions on a two dimensional grid. However, you can get close.

If you want to arrange from white->black and red->purple, you can define your distance function to treat differences in darkness as distance, as well as differences in hue value (no warping!). This will give you a set, four-corner-compatible sorting for your colors.

Now, anchor each of your colors to the four corners, like so, defining (0:0) as black, (1:1) as white, (0,1) as red (0 hue), and (1:0) as purple-red (350+ hue). Like so (let's say purple-red is purple for simplicity):

Now, you have two metrics of extremes: darkness and hue. But wait... if we rotate the box by 45 degrees...

Do you see it? No? The X and Y axes have aligned with our two metrics! Now all we need to do is divide each color's distance from white with the distance of black from white, and each color's distance from purple with the distance of red from purple, and we get our Y and X coordinates, respectively!

Let's add us a few more pens:

Now iterate over all the pens with O(n)^2, finding the closest distance between any pen and a final pen position, distributed uniformly through the rotated grid. We can keep a mapping of these distances, replacing any distances if the respective pen position has been taken. This will allow us to stick pens into their closest positions in polynomial time O(n)^3.

However, we're not done yet. HSV is 3 dimensional, and we can and should weigh the third dimension into our model too! To do this, we extend the previous algorithm by introducing a third dimension into our model before calculating closest distances. We put our 2d plane into a 3d space by intersecting it with the two color extremes and the horizontal line between white and black. This can be done simply by finding the midpoint of the two color extremes and nudging darkness slightly. Then, generate our pen slots fitted uniformly onto this plane. We can place our pens directly in this 3D space based off their HSV values - H being X, V being Y, and S being Z.

Now that we have the 3d representation of the pens with saturation included, we can once again iterate over the position of pens, finding the closest one for each in polynomial time.

There we go! Nicely sorted pens. If you want the result in an array, just generate the coordinates for each array index uniformly again and use those in order!

Now stop sorting pens and start making code!

Answer 3

As it was pointed out to you in some of the comments, you seem to be interested in finding one of the global minima of a discrete optimization problem. You might need to read up on that if you don't know much about it already.

Imagine that you have an error (objective) function that is simply the sum of distance(c1, c2) for all (c1, c2) pairs of adjacent pens. An optimal solution (arrangement of pens) is one whose error function is minimal. There might be multiple optimal solutions. Be aware that different error functions may give different solutions, and you might not be satisfied with the results provided by the simplistic error function I just introduced.

You could use an off-the-shelf optimizer (such as CPLEX or Gurobi) and just feed it a valid formulation of your problem. It might find an optimal solution. However, even if it does not, it may still provide a sub-optimal solution that is quite good for your eyes.

You could also write your own heuristic algorithm (such as a specialized genetic algorithm) and get a solution that is better than what the solver could find for you within the time and space limit it had. Given that your weapons seem to be input data, a function to measure color dissimilarity, and JavaScript, implementing a heuristic algorithm is probably the path that will feel most familiar to you.

---

My answer originally had no code with it because, as is the case with most real-world problems, there is no simple copy-and-paste solution for this question.

Doing this sort of computation using JavaScript is weird, and doing it on the browser is even weirder. However, because the author explicitly asked for it, here is a JavaScript implementation of a simple evolutionary algorithm hosted on CodePen.

Because of the larger input size than the 5x5 I originally demonstrated this algorithm with, how many generations the algorithm goes on for, and how slow code execution is, it takes a while to finish. I updated the mutation code to prevent mutations from causing the solution cost to be recomputed, but the iterations still take quite some time. The following solution took about 45 minutes to run in my browser through CodePen's debug mode.

Its objective function is slightly less than 2060 and was produced with the following parameters.

const SelectionSize = 100;
const MutationsFromSolution = 50;
const MutationCount = 5;
const MaximumGenerationsWithoutImprovement = 5;

It's worth pointing out that small tweaks to parameters can have a substantial impact on the algorithm's results. Increasing the number of mutations or the selection size will both increase the running time of the program significantly, but may also lead to better results. You can (and should) experiment with the parameters in order to find better solutions, but they will likely take even more compute time.

In many cases, the best improvements come from algorithmic changes rather than just more computing power, so clever ideas about how to perform mutations and recombinations will often be the way to get better solutions while still using a genetic algorithm.

Using an explicitly seeded and reproducible PRNG (rather than Math.random()) is great, as it will allow you to replay your program as many times as necessary for debugging and reproducibility proofs.

You might also want to set up a visualization for the algorithm (rather than just console.log(), as you hinted to) so that you can see its progress and not just its final result.

Additionally, allowing for human interaction (so that you can propose mutations to the algorithm and guide the search with your own perception of color similarity) may also help you to get the results you want. This will lead you to an Interactive Genetic Algorithm (IGA). The article J. C. Quiroz, S. J. Louis, A. Shankar and S. M. Dascalu, "Interactive Genetic Algorithms for User Interface Design," 2007 IEEE Congress on Evolutionary Computation, Singapore, 2007, pp. 1366-1373, doi: 10.1109/CEC.2007.4424630. is a good example of such approach.

Answer 4

If you could define a total ordering function between two colors that tell you which one is the 'darker' color, you can sort the array of colors using this total ordering function from dark to light (or light to dark).

You start at the top left with the first color in the sorted array, keep going diagonally across the grid and fill the grid with the subsequent elements. You will get a gradient filled rectangular grid where adjacent colors would be similar.

Do you think that would meet your objective?

You can change the look by changing the behavior of the total ordering function. For example, if the colors are arranged by similarity using a color map as shown below, you can define the total ordering as a traversal of the map from one cell to the next. By changing which cell gets picked next in the traversal, you can get different color-similar gradient grid fills.

Answer 5

I think there might be a simple approximate solution to this problem based on placing each color where it is the approximate average of the sorrounding colors. Something like:

C[j] ~ sum_{i=1...8}(C[i])/8

Which is the discrete Laplace operator i.e., solving this equation is equivalent to define a discrete harmonic function over the color vector space i.e., Harmonic functions have the mean-value property which states that the average value of the function in a neighborhood is equal to its value at the center.

In order to find a particular solution we need to setup boundary conditions i.e., we must fix at least two colors in the grid. In our case it looks convinient to pick 4 extrema colors and fix them to the corners of the grid.

One simple way to solve the Laplace's equation is the relaxation method (this amounts to solve a linear system of equations). The relaxation method is an iterative algorithm that solves one linear equation at a time. Of course in this case we cannot use a relaxation method (e.g., Gauss Seidel) directly because it is really a combinatorial problem more than a numercal problem. But still we can try to use relaxation to solve it.

The idea is the following. Start fixing the 4 corner colors (we will discuss about those colors later) and fill the grid with the bilinear interpolation of those colors. Then pick a random color C_j and compute the corresponding Laplacian color L_j i.e., the average color of sorrounding neighbors. Find the color closest to L_j from the set of input colors. If that color is different to C_j then replace C_j with it. Repeat the process until all colors C_j have been searched and no color replacements are needed (convergence critetia).

The function that find the closest color from input set must obey some rules in order to avoid trivial solutions (like having the same color in all neighbors and thus also in the center).

First, the color to find must be the closest to L_j in terms of Euclidian metric. Second, that color cannot be the same as any neighbor color i.e., exclude neighbors from search. You can see this match as a projection operator into the input set of colors.

It is expected that covergence won't be reached in the strict sense. So limiting the number of iterations to a large number is acceptable (like 10 times the number of cells in the grid). Since colors C_j are picked randomly, there might be colors in the input that were never placed in the grid (which corresponds to discontinuities in the harmonic function). Also there might be colors in the grid which are not from input (i.e., colors from initial interpolation guess) and there might be repeated colors in the grid as well (if the function is not a bijection).

Those cases must be addressed as special cases (as they are singularities). So we must replace colors from initial guess and repeated colors with that were not placed in the grid. That is a search sub-problem for which I don't have a clear euristic to follow beyond using distance function to guess the replacements.

Now, how to pick the first 2 or 4 corner colors. One possible way is to pick the most distinct colors based on Euclidean metric. If you treat colors as points in a vector space then you can perform regular PCA (Principal Component Analysis) on the point cloud. That amounts to compute the eigenvectors and corresponding eigenvalues of the covariance matrix. The eigenvector corresponding to the largest eigenvalue is a unit vector that points towards direction of greatest color variance. The other two eigenvectors are pointing in the second and third direction of greatest color variance in that order. The eigenvectors are orthogonal to each other and eigenvalues are like the "length" of those vectors in a sense. Those vectors and lengths can be used to determine an ellipsoid (egg shape surface) that approximately sorround the point cloud (let alone outliers). So we can pick 4 colors in the extrema of that ellipsoid as the boundary conditions of the harmonic function.

I haven't tested the approach, but my intuition ia that it should give you a good approximate solution if the input colors vary smoothly (the colors corresponds to a smooth surface in color vector space) otherwise the solution will have "singularities" which mean that some colors will jump abruptly from neighbors.

EDIT:

I have (partially) implemented my approach, the visual comparison is in the image below. My handling of singularities is quite bad, as you can see in the jumps and outliers. I haven't used your JS plumbing (my code is in C++), if you find the result useful I will try to write it in JS.

Answer 6

I would define a concept of color regions, that is, a group of colors where distance(P1, P2) <= tolerance. In the middle of such a region you would find the point which is closest to all others by average.

Now, you start with a presumably unordered grid of colors. The first thing my algorithm would do is to identify items which would fit together as color regions. By definition each region would fit well together, so we arrive to the second problem of interregion compatibility. Due to the very ordered manner of a region and the fact that into its middle we put the middle color, its edges will be "sharp", that is, varied. So, region1 and region2 might be much more compatible, if they are placed together from one side than the other side. So, we need to identify which side the regions are desirably glued together and if for some reason "connecting" those sides is impossible (for example region1 should be "above" region2, but, due to the boundaries and the planned positions of other regions), then one could "rotate" one (or both) the regions.

The third step is to check the boundaries between regions after the necessary rotations were made. Some repositioning of the items on the boundaries might still be needed.