Digital camera algorithms

Question

I'm working on a simple video device and I'd like to introduce some standard cool camera features. Amongst all I'd like to introduce

• Focus indicator
• Auto focus
• Auto exposure (ideal exposure time estimation)

Right now I'm looking for some examples, how these features can be implemented. Do you have any useful links?

EDIT : Ok, I will use standard CCD camera, which can provide me ~ 20fps in ~1MPix resolution. I'm planning to write it in C#, in case of performance issues, I'll use C++. I'll have lens + CCD camera + motor.

EDIT : I'd like to see some more detailed algorithm description. I'm sure some have to be taught in university courses, but I have troubles finding some. For focus indicator I've tried a primitive approach, but in some cases it failed.

 int verticalPoints = 0, horizontalPoints = 0;
 ///Calculate the vertical differences
 for (int x = 0; x < toAnalyze.Width; x++)
 {
     for (int y = 1; y < toAnalyze.Height; y++)
     {
        byte* pixel = (byte*)data.Scan0 + y * stride + x;
        verticalDiff += Math.Abs(*pixel - *(pixel - stride));;
     }         
  }
  verticalDiff /= toAnalyze.Width * (toAnalyze.Height-1);
  ///Calculate horizontal differences
  for (int y = 0; y < toAnalyze.Height; y++)
  {
     for (int x = 1; x < toAnalyze.Width; x++)
     {
        byte* pixel = (byte*)data.Scan0 + y * stride + x;
        horizontalDiff += Math.Abs(*pixel - *(pixel - 1));
     }
   }
   horizontalDiff /= (toAnalyze.Width-1) * toAnalyze.Height;
   ///And return the average value
   return(verticalDiff + horizontalDiff) / 2;

Thanks

Answer 1

Starting from the end, so to speak:

Auto-exposure is pretty simple: measure the light level and figure out how long of an exposure is needed for that average light to produce ~15-18% gray level. There are lots of attempts at improving that (usually by metering a number of sections of the picture separately, and processing those results), but that's the starting point.

There are two separate types of autofocus. Most video cameras use one based on detecting contrast -- look at the input from the sensor, and when the differences between adjacent pixels are maximized, you consider that "in focus."

Contrast detection autofocus does make it a bit difficult to do focus indication though -- in particular, you never really know when you've achieved maximum contrast until the contrast starts to fall again. When you're doing autofocus, you focus until you see a peak and then see it start to fall again, and then drive it back to where it was highest. For manual focus with an indicator, you can't recognize maximum contrast until it starts to fall again. The user would have to follow roughly the same pattern, moving past best focus, then back to optimum.

Alternatively, you could use phase detection. This uses the alignment of the "pictures" coming through two prisms, much like the split-image viewfinders that were used in many (most?) SLRs before autofocus came into use.

Answer 2

Just to inform you. I am working on a professional forensic 5 Megapixel digital camera software in WPF. In DotNet not C++. There are some threading issus to know but it works perfectly fast. More performant because GPU is used.

Jerry did a good work with his answer. Focus detection is "Contrast detection based on time / frames". Logic is simple, to keep it performant it is not easy. Auto Focus detection

To check the exposure time, it is easy if you have created the histogram of image. Image histogram In any case you need to do it for

• Red channel
• Green channel
• Blue channel
• Gain
• Exposure time

This mix makes it a bit more complicated because you also can use Color gain channels to increase brightness of image. RGB image digital. Luminance can have the same result like with "Gain" and "Exposure" time.

If you calculate the exposure time automatically, keep good in mind that you need a frame to calculate it and as smaller the exposure time as more frames you will get. That means, if you want to have a good algorithm, always try to have a very small exposure time and increase it slowly. Not use a linear algorithm where you decrease the value slowly.

There are also more methodes for digital cameras like Pixel Binning Pixel Binning to increase framerate to get quick focus results.

Here is a sample of how focus can work to generate a focus intensity image :

 Private Sub GetFocusValue(ByRef C1 As Color, ByVal LCol1 As List(Of Color), ByVal LCol2 As List(Of Color), ByVal AmplifierPercent As Single)
        Dim MaxDiff1 As Integer = 0
        Dim MaxDiff2 As Integer = 0
        Dim Factor As Single = 0
        Dim D As Integer

        Dim LR1 As New List(Of Integer)
        Dim LR2 As New List(Of Integer)
        Dim LG1 As New List(Of Integer)
        Dim LG2 As New List(Of Integer)
        Dim LB1 As New List(Of Integer)
        Dim LB2 As New List(Of Integer)

        For Each C As Color In LCol1
            LR1.Add(C.R)
            LG1.Add(C.G)
            LB1.Add(C.B)
        Next


        For Each C As Color In LCol2
            LR2.Add(C.R)
            LG2.Add(C.G)
            LB2.Add(C.B)
        Next



        MaxDiff1 = Me.GetMaxDiff(LR1)
        MaxDiff1 = Math.Max(MaxDiff1, Me.GetMaxDiff(LG1))
        MaxDiff1 = Math.Max(MaxDiff1, Me.GetMaxDiff(LB1))


        MaxDiff2 = Me.GetMaxDiff(LR2)
        MaxDiff2 = Math.Max(MaxDiff2, Me.GetMaxDiff(LG2))
        MaxDiff2 = Math.Max(MaxDiff2, Me.GetMaxDiff(LB2))



        If MaxDiff1 > MaxDiff2 Then
            D = MaxDiff1 - MaxDiff2
            Factor = D / 255
            Factor = Factor / (AmplifierPercent / 100)
            Factor = Math.Min(Factor, 1)
            Factor = 1 - Factor 'invert result
            'TB.Math.Swap(MaxDiff1, MaxDiff2)
            'Factor = 255 'the original BM1 is better
        Else
            D = MaxDiff2 - MaxDiff1
            Factor = D / 255
            Factor = Factor * (AmplifierPercent / 100)
            Factor = Math.Min(Factor, 1)
            'Factor = 0 'the BM2 is better
        End If
        Factor = Factor * 255



        C1 = Color.FromArgb(Convert.ToByte(Factor), C1.R, C1.G, C1.B)


    End Sub

Answer 3

The AForge.net has alot of stuff for doing image processing , including edge detection and convolution filters. Another (larger) library you might want to look at is OpenCV but only has wrappers for .net, where as AForge is written in c# directly