2

Hi guys I’ve thinking about this question:

I know that we use Fourier transform to get into frequency domain to process the image.

I read the text book, it said that when we are done with processing the image in the Fourier domain we have to invert it back to get processed image.

And the textbook taught to get the real part of the inverse.

However, when I go through the OpenCv tutorial, no matter if using OpenCV or NumPy version, eventually they use magnitude (for OpenCV) or np.abs (for NumPy).

For OpenCV, the inverse returns two channels which contain the real and imaginary components. When I took the real part of the inverse, I got a totally weird image.

May somebody who knows the meaning behind all of this:

  1. Why using magnitude or abs to get processed image?

  2. What’s wrong with textbook instruction (take the real part of inverse)?

Cris Luengo
  • 55,762
  • 10
  • 62
  • 120
H_E_A_D
  • 101
  • 1
  • 1
  • 8
  • you can only use real numbers in image, when you take inverse fourier transform it might have complex numbers as output, so you need to get real part of the inverse FT and display it as an image – user8190410 Nov 12 '18 at 17:19
  • https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_transforms/py_fourier_transform/py_fourier_transform.html. – H_E_A_D Nov 12 '18 at 17:27
  • The link calculate the magnitude and the result looks great – H_E_A_D Nov 12 '18 at 17:27

1 Answers1

4

The textbook is right, the tutorial is wrong.

A real-valued image has a complex conjugate symmetry in the Fourier domain. This means that the FFT of the image will have a specific symmetry. Any processing that you do must preserve this symmetry if you want the inverse transform to remain real-valued. If you do this processing wrong, then the inverse transform will be complex-valued, and probably non-sensical.

If you preserve the symmetry in the Fourier domain properly, then the imaginary component of the inverse transform will be nearly zero (likely different from zero because of numerical imprecision). Discarding this imaginary component is the correct thing to do. Computing the magnitude will yield the same result, except all negative values will become positive (note some filters are meant to produce negative values, such as derivative filters), and at an increased computational cost.

For example, a convolution is a multiplication in the Fourier domain. The filter in the Fourier domain must be real-valued and symmetric around the origin. Often people will confuse where the origin is in the Fourier domain, and multiply by a filter that is seems symmetric, but actually is shifted with respect to the origin making it not symmetric. This shift introduces a phase change of the inverse transform (see the shift property of the Fourier transform). The magnitude of the inverse transform is not affected by the phase change, so taking the magnitude of this inverse transform yields an output that sort of looks OK, except if one expects to see negative values in the filter result. It would have been better to correctly understand the FFT algorithm, create a properly symmetric filter in the Fourier domain, and simply keep the real part of the inverse transform.

Nonetheless, some filters are specifically designed to break the symmetry and yield a complex-valued filter output. For example the Gabor filter has an even (symmetric) component and an odd (anti-symmetric) component. The even component yields a real-valued output, the odd component yields an imaginary-valued output. In this case, it is the magnitude of the complex value that is of interest. Likewise, a quadrature filter is specifically meant to produce a complex-valued output. From this output, the analytic signal (or its multi-dimensional extension, the monogenic signal), both the magnitude and the phase are of interest, for example as used in the phase congruency method of edge detection.

Looking at the linked tutorial, it is the line 

fshift[crow-30:crow+30, ccol-30:ccol+30] = 0

which generates the Fourier-domain filter and applies it to the image (it is equivalent to multiplying by a filter with 1s and 0s). This tutorial correctly computes the origin of the Fourier domain (though for Python 3 you would use crow,ccol = rows//2 , cols//2 to get the integer division). But the filter above is not symmetric around that origin. In Python, crow-30:crow+30 indicates 30 pixels to the left of the origin, and only 29 pixels to the right (the right bound is not included!). The correct filter would be:

fshift[crow-30:crow+30+1, ccol-30:ccol+30+1] = 0

With this filter, the inverse transform is purely real (imaginary component has values in the order of 1e-13, which is numerical errors). Thus, it is now possible (and correct) to replace img_back = np.abs(img_back) with img_back = np.real(img_back).

Cris Luengo
  • 55,762
  • 10
  • 62
  • 120
  • So you mean I take the real part of inverse and get the wrong image is because there’s something wrong with my filter? Should I need to shift filter as well before doing multiplication ？ Thanks in advance! – H_E_A_D Nov 12 '18 at 17:36
  • I think I forgot shifting the kernel before multiplication ,wouldn’t this cause the result not symmetric? – H_E_A_D Nov 12 '18 at 17:52
  • 1
    @kris: it depends on how you created the kernel. In the tutorial, they used `fftshift` on the transform, which means that the origin is in the middle of the image. Your kernel must be designed to have the origin in the same location. If your kernel is the result of an FFT of a kernel made in the spatial domain, you should apply `ifftshift` to the spatial-domain kernel, then apply `fft` to the image and the shifted kernel, then multiply, then do the inverse transform (no `fftshift` in the Fourier domain). – Cris Luengo Nov 12 '18 at 17:59
  • One more question: you mentioned that The filter in the Fourier domain must be real-valued and symmetric around the origin. What if the filter is not symmetric and the result of multiplication of image and filter would not be symmetric,right? So in this case we have to take both part and imaginary part? If yes,it just got me wondering why calculate their magnitude,wouldn't the result of operation would be different than the what we expect since we normally just take the real part? – H_E_A_D Nov 13 '18 at 02:40
  • 1
    @kris: If the filter is not symmetric, then the inverse transform will be a complex-valued function. Some filters are designed this way (see the paragraph I added to the answer). It depends on the filter and how it is designed, what one does with the complex output. The discussion about symmetric filters relates to filters that are meant to keep the real-valued image being real-valued, such as low-pass, high-pass or band-pass filters. – Cris Luengo Nov 13 '18 at 05:19
  • So we have to use magnitude to get the correct processed image? I just have no idea why taking the magnitude wold gives the correct result.... Can you help me out with this ,Thanks! – H_E_A_D Nov 13 '18 at 06:15
  • @kris: you have to take the magnitude if your filter is a Gabor filter. Otherwise, no, you don't take the magnitude. In general, filters are symmetric and yield a real-valued result. Take the real component and discard the imaginary component. Complex-valued filters are rare and specialized things. If you use them, you should know what you're doing and what you want to do with the complex output. – Cris Luengo Nov 13 '18 at 06:20
  • 1
    The error in the OpenCV documentation has been corrected: https://github.com/opencv/opencv/issues/13152 – Cris Luengo Nov 16 '18 at 02:52
  • Wow,Really nice ! – H_E_A_D Nov 16 '18 at 04:22
  • Can I maybe ask you one more question about Fourier transform: Why do we use Fourier transform? Is that because the function not periodic?What happen if function use Fourier transform ,would the function become periodic in frequency domain? Thanks! – H_E_A_D Nov 28 '18 at 07:37
  • @kris: The Fourier transform is a great tool to analyze linear systems (the ones you can define by a convolution), and to *define* them. For example quadrature filters are defined through their properties in the FD. In the digital world you can also use it to cheaply compute convolutions with large kernels. The DFT makes both the image and the FD periodic. I’m not sure this answers your question, it’s hard to summarize a whole book in 400 characters... :) – Cris Luengo Nov 28 '18 at 13:34
  • yeah I want know the basic purpose of Fourier transform: (1)Does Fourier transform exist because lots of function are not periodic? (2)And then after transformation,the function would be periodic and extend to infinite in frequency domain? Those questions above are all I want to make sure :) And again,Thanks! your answer is just so clear ! – H_E_A_D Nov 29 '18 at 08:12
  • 1
    @kris: the FT is a continuous-domain construct that requires an infinitely long signal. Sampling this signal leads to a periodic frequency domain. Now you have the DTFT (discrete time FT). Windowing the signal (when we measure a signal we don’t have infinitely long data) means we cannot apply the FT any more. So instead we assume periodicity of the signal, repeating it infinitely. Now we do have something we can compute the FT of. The periodicity leads to a discrete frequency domain. Now you have a DFT (discrete FT). The FFT computes the DFT. – Cris Luengo Nov 29 '18 at 14:17
  • Wow okay Thanks, I’m new to this but willing to spend time learning.I’m thinking about if I can have your email so that I can get some help when I do need.If you are willing to do so ,I would leave mine in the next comment.Thanks! – H_E_A_D Nov 30 '18 at 05:58
  • 1
    @kris: The [DSP Stack Exchange](https://dsp.stackexchange.com) is a good place to ask questions about Fourier analysis and many other things. – Cris Luengo Nov 30 '18 at 06:26
  • Okay Thanks I want to focus on image processing through. – H_E_A_D Nov 30 '18 at 09:01