I have a list of colours and want to merge the two "closest" (most similar) by taking their average. To do this I need a numerical distance function between colours. I could treat RGB as a 3D vector for distance, however this may not be perceptively best. E.g. the human eye apparently notices contrast in luminance far better than hue. For that matter, averaging may not be correct either. The speed of this calculation is also a concern.
Before jumping in to various colour spaces and color difference, I'd like to know if there is a well know method that loosely balances visual accuracy and performance. Is the 3D distance the way to go?