Get convolution matrix from image samples?

Question

Im trying to decompile the convolution matrix for the filters on the Motorola Gallery App

Im using the following code to read the pixel data:

public static void main(String[] foo) {
new JavaWalkBufferedImageTest1();
}

public void printPixelARGB(int pixel) {
int alpha = (pixel >> 24) & 0xff;
int red = (pixel >> 16) & 0xff;
int green = (pixel >> 8) & 0xff;
int blue = (pixel) & 0xff;
System.out.println("argb: " + alpha + ", " + red + ", " + green + ", " + blue);
}

private void marchThroughImage(BufferedImage image) {
int w = image.getWidth();
int h = image.getHeight();
System.out.println("width, height: " + w + ", " + h);

for (int i = 0; i < h; i++) {
  for (int j = 0; j < w; j++) {
    System.out.println("x,y: " + j + ", " + i);
    int pixel = image.getRGB(j, i);
    printPixelARGB(pixel);
    System.out.println("");
  }
}
}

  public JavaWalkBufferedImageTest1() {
  try {
  // get the BufferedImage, using the ImageIO class
  BufferedImage image = 
    ImageIO.read(this.getClass().getResource("WhiteSpot.jpg"));
  marchThroughImage(image);
} catch (IOException e) {
  System.err.println(e.getMessage());
}
}

It gives the desired output. But this is not leading me anywhere into finding the matrix.

How do I modify the code so that I can input 2 image files, Original & Filtered. And get the convolution matrix. Or is there an online tool that I can use, where I upload multiple Original & Filtered samples, and get the convolution matrix?

Answer 1

I would approach this problem (will work if only convolution was applied and only once!) like this:

1. first you need to find matrix size.

   So you can loop through all possible matrix sizes or use a big one expecting zero values inside. That can be slow but you can try to estimate matrix size from the bluryness of sharp intensity edges (to how many pixels is the color bleeded).

2. for tested (big) matrix size try to find zero values

   So for each tested matrix value form 2 arrays. One with few samples of pixels from convoluted image and second with pixels (shifted by tested matrix position) from original image. Now compute correlation coefficient between the two and if no significant correlation present you can assume the value in matrix is zero.

3. solve the remaining nonzero values in matrix

   You can do this algebraically (form as many equation as many nonzero values you have (do not forget to choose pixels with different color). Or you can do search to minimize error/distance of convoluted output by your matrix and the real convoluted output. You can use something like mine approx class in C++ for this but if Your matrix is big then this will take a lot of time.

[Notes]

If more then just single convolution is applied to image then this will most likely not work.

Convolution matrices are mostly square and symmetric around mid value so you can compute the quarter of values and mirror the rest ... speeding up the computation.

Also see Dealing with filters and colour's which is similar question.

Bullet #2 can be used to detect the matrix size. As usually the max nonzero values are in the central cross of matrix.

0 0 2 0 0
0 1 2 1 0
2 2 3 2 2
0 1 2 1 0
0 0 2 0 0

Values 2 are the central cross and value 3 is the mid value of convolution matrix. So start computing the #2 correlations from mid point in x and y direction. And when hit zero value you are most likely at the edge of convolution matrix. So you can use that as matrix size (unless the matrix is some weird filter ...).

Also each color channel can have its own convolution matrix so may be you should do this separately per channel (and convolution can be done in any color space not just in RGB).

There might be other approaches for this (my guts tells me this should be solvable by PCA)

If you got black area in the original image then you can use that to find the values of matrix more easily (it will significantly ease up the algebraic approach)