1.
There are many different methods for comparing colors. If you want perceptual dissimilarity, check out the comments of other users. If you just want something simple that gives an indication: use this code:
int rgb1 = ...
int rgb2 = ...
int r, g, b;
r = (rgb1 >> 16) & 0x000000ff;
g = (rgb1 >> 8) & 0x000000ff;
b = rgb1 & 0x000000ff;
r -= (rgb2 >> 16) & 0x000000ff;
g -= (rgb2 >> 8) & 0x000000ff;
b -= rgb2 & 0x000000ff;
double answer = (r*r+g*g+b*b)/(255*255*3.0);
It calculates a euclidean distance and scales it to range from 0 to 1.
2.
Just as there are many different ways to compare colors, there are many ways to combine colors. It depends on what you want.
You could take the average but that's not what you want. As it would darken the resulting color.
EDIT: here is the code to take the average:
r = (rgb1 >> 16) & 0x000000ff;
g = (rgb1 >> 8) & 0x000000ff;
b = rgb1 & 0x000000ff;
r += (rgb2 >> 16) & 0x000000ff;
g += (rgb2 >> 8) & 0x000000ff;
b += rgb2 & 0x000000ff;
int average = ((r/2)<<16)+((g/2)<<8)+(b/2);
If you are somewhat mathematically inclined think about what behavior you actually look for, and try to put it in a formula (or rather 3 formulas, one for red, one for green, and one for blue)
It shouldn't be hard to convert the formula to java code.
You can put individual r, g and b values back into a single rgb value using
int rgb = (r<<16) | (g<<8) | b
Edit: notice that computers work with `red GREEN blue´, where yellow is a combination of green and blue, not the other way around. Conversion can be done, but I have not done it yet. Perhaps conversion between rgb and ryb color spaces may be of help.