7

I'm using Winforms. In my form I have a picturebox that displays a black and white image. I also have a button that if you click it, the button will remove spots / dots on the images. When the image dimension is not big it removes the spots quickly. If the image is large it takes a while. Also sometimes this function removes some of the words from the image it thinks its a spot. How can I improve the performance of this function and more accurately remove the spots or dots basically despeckle the image?

Update Upon research I found this library that seemed promising for this question:

http://www.aforgenet.com/framework/docs/html/cdf93487-0659-e371-fed9-3b216efb6954.htm

Link for spotted image: http://www.filedropper.com/testing-image3

Image Example

Note image in link has a bigger version of this:

enter image description here

Image Information

The thing to note here is that it is a black and white image - Bit Depth 1

enter image description here

My Code

    private int[] mask = new int[9];
    private void remove_spot_btn_Click(object sender, EventArgs e)
    {
        Bitmap img = new Bitmap(pictureBox1.Image);
        Color c;

        for (int ii = 0; ii < img.Width; ii++)
        {
            for (int jj = 0; jj < img.Height; jj++)
            {

                if (ii - 1 >= 0 && jj - 1 >= 0)
                {
                    c = img.GetPixel(ii - 1, jj - 1);
                    mask[0] = Convert.ToInt16(c.R);
                }
                else
                {
                    mask[0] = 0;
                }

                if (jj - 1 >= 0 && ii + 1 < img.Width)
                {
                    c = img.GetPixel(ii + 1, jj - 1);
                    mask[1] = Convert.ToInt16(c.R);
                }
                else
                    mask[1] = 0;

                if (jj - 1 >= 0)
                {
                    c = img.GetPixel(ii, jj - 1);
                    mask[2] = Convert.ToInt16(c.R);
                }
                else
                    mask[2] = 0;

                if (ii + 1 < img.Width)
                {
                    c = img.GetPixel(ii + 1, jj);
                    mask[3] = Convert.ToInt16(c.R);
                }
                else
                    mask[3] = 0;

                if (ii - 1 >= 0)
                {
                    c = img.GetPixel(ii - 1, jj);
                    mask[4] = Convert.ToInt16(c.R);
                }
                else
                    mask[4] = 0;

                if (ii - 1 >= 0 && jj + 1 < img.Height)
                {
                    c = img.GetPixel(ii - 1, jj + 1);
                    mask[5] = Convert.ToInt16(c.R);
                }
                else
                    mask[5] = 0;

                if (jj + 1 < img.Height)
                {
                    c = img.GetPixel(ii, jj + 1);
                    mask[6] = Convert.ToInt16(c.R);
                }
                else
                    mask[6] = 0;


                if (ii + 1 < img.Width && jj + 1 < img.Height)
                {
                    c = img.GetPixel(ii + 1, jj + 1);
                    mask[7] = Convert.ToInt16(c.R);
                }
                else
                    mask[7] = 0;
                c = img.GetPixel(ii, jj);
                mask[8] = Convert.ToInt16(c.R);
                Array.Sort(mask);
                int mid = mask[4];
                img.SetPixel(ii, jj, Color.FromArgb(mid, mid, mid));
            }
        }

        pictureBox1.Image = img;
        MessageBox.Show("Complete");
    }
taji01
  • 2,527
  • 8
  • 36
  • 80
  • 4
    Please upload your image directly to the question with the "Image" button. The website you linked is scary to even look at. – KDecker Mar 02 '17 at 20:09
  • 5
    `GetPixel` and `SetPixel` are notoriously slow. You can get direct access to a `Bitmap`'s pixel data as an array by using [Bitmap.LockBits](https://msdn.microsoft.com/en-us/library/5ey6h79d(v=vs.110).aspx). – adv12 Mar 02 '17 at 20:10
  • I recommend studying the example included in the documentation I linked to. – adv12 Mar 02 '17 at 21:50
  • The image was to big. It wouldn't let me upload it here sorry. @KDecker – taji01 Mar 09 '17 at 19:33
  • You simply need to resize it and then upload it. – KDecker Mar 09 '17 at 20:04
  • @KDecker Thank you for the suggestion. I made the suggested changes. – taji01 Mar 09 '17 at 21:33
  • Thanks guys! This was a tough decision both of the answers was good, but using AForge.NET median filter did the trick. It helped Despeckle the images fast. – taji01 Mar 10 '17 at 22:49

5 Answers5

4

As also mentioned in comments, to change pixels in a Bitmap with better performance rather than SetPixel, you can use Bitmap.LockBits method to access bitmap data.

To make your code faster with minimal changes, You can create a class which encapsulates fast access to bitmap data using LockBits and create a GetPixel and a SetPixel method for the class.


Note: The answer just tries to make your code faster by applying minimal changes. It doesn't apply any enhancement in your algorithm for better noise reduction.
Example

For example, I used a class which is written by Vano Maisuradze with small changes (I removed unnecessary try/catch blocks from code). The class is using LockBits method and provides fast version of GetPixel and SetPixel methods.

Then your code should be changed to:

var bmp = new Bitmap(pictureBox1.Image);
var img = new LockBitmap(bmp);
img.LockBits();
Color c;
//...
//...
//...
img.UnlockBits();
pictureBox1.Image = bmp;
MessageBox.Show("Complete");

Here is the implementation of the class:

public class LockBitmap
{
    Bitmap source = null;
    IntPtr Iptr = IntPtr.Zero;
    BitmapData bitmapData = null;

    public byte[] Pixels { get; set; }
    public int Depth { get; private set; }
    public int Width { get; private set; }
    public int Height { get; private set; }

    public LockBitmap(Bitmap source)
    {
        this.source = source;
    }

    /// <summary>
    /// Lock bitmap data
    /// </summary>
    public void LockBits()
    {
        // Get width and height of bitmap
        Width = source.Width;
        Height = source.Height;

        // get total locked pixels count
        int PixelCount = Width * Height;

        // Create rectangle to lock
        Rectangle rect = new Rectangle(0, 0, Width, Height);

        // get source bitmap pixel format size
        Depth = System.Drawing.Bitmap.GetPixelFormatSize(source.PixelFormat);

        // Check if bpp (Bits Per Pixel) is 8, 24, or 32
        if (Depth != 8 && Depth != 24 && Depth != 32)
        {
            throw new ArgumentException("Only 8, 24 and 32 bpp images are supported.");
        }

        // Lock bitmap and return bitmap data
        bitmapData = source.LockBits(rect, ImageLockMode.ReadWrite, 
                                     source.PixelFormat);

        // create byte array to copy pixel values
        int step = Depth / 8;
        Pixels = new byte[PixelCount * step];
        Iptr = bitmapData.Scan0;

        // Copy data from pointer to array
        Marshal.Copy(Iptr, Pixels, 0, Pixels.Length);
    }

    /// <summary>
    /// Unlock bitmap data
    /// </summary>
    public void UnlockBits()
    {
        // Copy data from byte array to pointer
        Marshal.Copy(Pixels, 0, Iptr, Pixels.Length);

        // Unlock bitmap data
        source.UnlockBits(bitmapData);
    }

    /// <summary>
    /// Get the color of the specified pixel
    /// </summary>
    /// <param name="x"></param>
    /// <param name="y"></param>
    /// <returns></returns>
    public Color GetPixel(int x, int y)
    {
        Color clr = Color.Empty;

        // Get color components count
        int cCount = Depth / 8;

        // Get start index of the specified pixel
        int i = ((y * Width) + x) * cCount;

        if (i > Pixels.Length - cCount)
            throw new IndexOutOfRangeException();

        if (Depth == 32) // For 32 bpp get Red, Green, Blue and Alpha
        {
            byte b = Pixels[i];
            byte g = Pixels[i + 1];
            byte r = Pixels[i + 2];
            byte a = Pixels[i + 3]; // a
            clr = Color.FromArgb(a, r, g, b);
        }
        if (Depth == 24) // For 24 bpp get Red, Green and Blue
        {
            byte b = Pixels[i];
            byte g = Pixels[i + 1];
            byte r = Pixels[i + 2];
            clr = Color.FromArgb(r, g, b);
        }
        if (Depth == 8)
        // For 8 bpp get color value (Red, Green and Blue values are the same)
        {
            byte c = Pixels[i];
            clr = Color.FromArgb(c, c, c);
        }
        return clr;
    }

    /// <summary>
    /// Set the color of the specified pixel
    /// </summary>
    /// <param name="x"></param>
    /// <param name="y"></param>
    /// <param name="color"></param>
    public void SetPixel(int x, int y, Color color)
    {
        // Get color components count
        int cCount = Depth / 8;

        // Get start index of the specified pixel
        int i = ((y * Width) + x) * cCount;

        if (Depth == 32) // For 32 bpp set Red, Green, Blue and Alpha
        {
            Pixels[i] = color.B;
            Pixels[i + 1] = color.G;
            Pixels[i + 2] = color.R;
            Pixels[i + 3] = color.A;
        }
        if (Depth == 24) // For 24 bpp set Red, Green and Blue
        {
            Pixels[i] = color.B;
            Pixels[i + 1] = color.G;
            Pixels[i + 2] = color.R;
        }
        if (Depth == 8)
        // For 8 bpp set color value (Red, Green and Blue values are the same)
        {
            Pixels[i] = color.B;
        }
    }
}
Reza Aghaei
  • 120,393
  • 18
  • 203
  • 398
  • Thank you Reza, I see how you used LockBits from this class. It did make my code a lot faster, but i also tried Simons method too and it seems like using AForge library might be the way to go due to its speed. Any thoughts? – taji01 Mar 07 '17 at 18:18
  • 2
    Hi @taji01, You're welcome. I've not used AForge but as far as I know it's one of the best open source libraries in this field in .NET. So probably it would be a better choice than writing your own code. :) [AForge.NET](http://www.aforgenet.com/framework/) is an open source C# framework designed for developers and researchers in the fields of Computer Vision and Artificial Intelligence - image processing, neural networks, genetic algorithms, fuzzy logic, machine learning, robotics, etc. – Reza Aghaei Mar 07 '17 at 21:06
4

As you found out, using AForge.NET is a good idea (you just have to add it as a nuget). I suggest you use its Median filter which is often used for denoising (see Median Filter in wikipedia).

AForge needs 24bpp RGB images, so you need to convert it first in your sample case, but here is an example of code that seems to work quite well on it:

  // load the file as 24bpp RGB
  using (var bmp = LoadForFiltering(@"C:\temp\Testing-Image3.tif"))
  {
      var filter = new Median();

      // run the filter 
      filter.ApplyInPlace(bmp);

      // save the file back (here, I used png as the output format)
      bmp.Save(@"C:\temp\Testing-Image3.png");
  }


  private static Bitmap LoadForFiltering(string filePath)
  {
      var bmp = (Bitmap)Bitmap.FromFile(filePath);
      if (bmp.PixelFormat == PixelFormat.Format24bppRgb)
          return bmp;

      try
      {
          // from AForge's sample code
          if (bmp.PixelFormat == PixelFormat.Format16bppGrayScale || Bitmap.GetPixelFormatSize(bmp.PixelFormat) > 32)
              throw new NotSupportedException("Unsupported image format");

          return AForge.Imaging.Image.Clone(bmp, PixelFormat.Format24bppRgb);
      }
      finally
      {
          bmp.Dispose();
      }
  }

If you really need high performance, then you could go for NVidia CUDA/NPP (using directly the GPU) for example but this is more work, not directly supported from C# (and requires an NVidia card of course). Related question here: 2D CUDA median filter optimization and a white paper on CUDA here : Image Processing and Video Algorithms with CUDA

Community
  • 1
  • 1
Simon Mourier
  • 132,049
  • 21
  • 248
  • 298
  • Hey Instead of saving it, how do i load it back to picturebox? i tried `pictureBox1.Image = bmp;` It throws an error. `An unhandled exception of type 'System.ArgumentException' occurred in System.Drawing.dll` – taji01 Mar 06 '17 at 06:04
  • 2
    Never mind all i had to do was this `pictureBox1.Image = AForge.Imaging.Image.Clone(bmp, PixelFormat.Format24bppRgb);` – taji01 Mar 06 '17 at 06:17
1

Your code is taking the median value of 9 nearby pixels and effectively just blurring. This is not a good noise-reduction algorithm-- it is more of a blur algorithm. Investigate which noise-reduction algorithm you need for your solution (depends on the kind of noise you have) and go from there.

Tim
  • 5,940
  • 1
  • 12
  • 18
1

I have worked on my own solution, so for anyone who would like a simple Despeckle function that works without dependencies. The downside however is the pixels that are connected to the letters are not removed, and some of the smaller letters are seen as spots.

The result I get with "maxSpotSize = 10" is this enter image description here

The way it works is by using a fill on every pixel and measuring its area size (to the maxSpotSize limit you put it on), for speed reasons it only works with black/white images, but you could modify it to support a more complex fill with a tolerance.

LockBitmap

First of all, (mentioned by Reza Aghaei) make use of a LockBitmap class in your project to get access to the pixels in an efficient way. Put this code inside a new class and change the namespace to your project's namespace:

using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;

namespace //<YOUR NAMESPACE>//
{
    public class LockBitmap : IDisposable
    {

        public void Dispose()
        {
            UnlockBits();
        }

        public Bitmap source = null;
        IntPtr Iptr = IntPtr.Zero;
        BitmapData bitmapData = null;

        public byte[] Pixels { get; set; }
        public int Depth { get; private set; }
        public int Width { get; private set; }
        public int Height { get; private set; }

        public LockBitmap(Bitmap source)
        {
            this.source = source;
            LockBits();
        }

        /// <summary>
        /// Lock bitmap data
        /// </summary>
        private void LockBits()
        {
            try
            {
                // Get width and height of bitmap
                Width = source.Width;
                Height = source.Height;

                // get total locked pixels count
                int PixelCount = Width * Height;

                // Create rectangle to lock
                Rectangle rect = new Rectangle(0, 0, Width, Height);

                // get source bitmap pixel format size
                Depth = System.Drawing.Bitmap.GetPixelFormatSize(source.PixelFormat);

                // Check if bpp (Bits Per Pixel) is 8, 24, or 32
                if (Depth != 8 && Depth != 24 && Depth != 32)
                {
                    throw new ArgumentException("Only 8, 24 and 32 bpp images are supported.");
                }

                // Lock bitmap and return bitmap data
                bitmapData = source.LockBits(rect, ImageLockMode.ReadWrite,
                                             source.PixelFormat);

                // create byte array to copy pixel values
                int step = Depth / 8;
                Pixels = new byte[PixelCount * step];
                Iptr = bitmapData.Scan0;

                // Copy data from pointer to array
                Marshal.Copy(Iptr, Pixels, 0, Pixels.Length);
            }
            catch (Exception ex)
            {
                throw ex;
            }
        }

        /// <summary>
        /// Unlock bitmap data
        /// </summary>
        private void UnlockBits()
        {
            try
            {
                // Copy data from byte array to pointer
                Marshal.Copy(Pixels, 0, Iptr, Pixels.Length);

                // Unlock bitmap data
                source.UnlockBits(bitmapData);
            }
            catch (Exception ex)
            {
                throw ex;
            }
        }

        /// <summary>
        /// Get the color of the specified pixel
        /// </summary>
        /// <param name="x"></param>
        /// <param name="y"></param>
        /// <returns></returns>
        public Color GetPixel(int x, int y)
        {

            // Get color components count
            int cCount = Depth / 8;

            // Get start index of the specified pixel
            int i = ((y * Width) + x) * cCount;

            if (i > Pixels.Length - cCount)
                throw new IndexOutOfRangeException();

            if (Depth == 32) // For 32 bpp get Red, Green, Blue and Alpha
            {
                byte b = Pixels[i];
                byte g = Pixels[i + 1];
                byte r = Pixels[i + 2];
                byte a = Pixels[i + 3]; // a
                return Color.FromArgb(a, r, g, b);
            }
            if (Depth == 24) // For 24 bpp get Red, Green and Blue
            {
                byte b = Pixels[i];
                byte g = Pixels[i + 1];
                byte r = Pixels[i + 2];
                return Color.FromArgb(r, g, b);
            }
            if (Depth == 8)
            // For 8 bpp get color value (Red, Green and Blue values are the same)
            {
                byte c = Pixels[i];
                return Color.FromArgb(c, c, c);
            }

            return Color.Empty;
        }

        /// <summary>
        /// Get the red color of the specified pixel
        /// </summary>
        /// <param name="x"></param>
        /// <param name="y"></param>
        /// <returns></returns>
        public int GetRedColor(int x, int y)
        {

            // Get color components count
            int cCount = Depth / 8;

            // Get start index of the specified pixel
            int i = ((y * Width) + x) * cCount;

            if (i > Pixels.Length - cCount)
                throw new IndexOutOfRangeException();

            if (Depth == 32 || Depth == 24) // For 32 bpp get Red, Green, Blue and Alpha
            {
                byte r = Pixels[i + 2];
                return r;
            }
            if (Depth == 8)
            // For 8 bpp get color value (Red, Green and Blue values are the same)
            {
                byte c = Pixels[i];
                return c;
            }

            return 0;
        }

        /// <summary>
        /// Set the color of the specified pixel
        /// </summary>
        /// <param name="x"></param>
        /// <param name="y"></param>
        /// <param name="color"></param>
        public void SetPixel(int x, int y, Color color)
        {
            // Get color components count
            int cCount = Depth / 8;

            // Get start index of the specified pixel
            int i = ((y * Width) + x) * cCount;

            if (Depth == 32) // For 32 bpp set Red, Green, Blue and Alpha
            {
                Pixels[i] = color.B;
                Pixels[i + 1] = color.G;
                Pixels[i + 2] = color.R;
                Pixels[i + 3] = color.A;
            }
            if (Depth == 24) // For 24 bpp set Red, Green and Blue
            {
                Pixels[i] = color.B;
                Pixels[i + 1] = color.G;
                Pixels[i + 2] = color.R;
            }
            if (Depth == 8)
            // For 8 bpp set color value (Red, Green and Blue values are the same)
            {
                Pixels[i] = color.B;
            }
        }

    }
}

Despeckle code

Now you can put this code somewhere in a static class, so it works like a extension method on a image or bitmap.

    public static Image Despeckle(this Image image, int maxSpotSize)
    {
        return ((Bitmap)image).Despeckle(maxSpotSize);
    }
    public static Bitmap Despeckle(this Bitmap bitmap, int maxSpotSize)
    {
        if (maxSpotSize == 0) { return bitmap; }

        Bitmap outputBitmap = new Bitmap(bitmap);

        using (LockBitmap outputLockBitmap = new LockBitmap(outputBitmap))
        {
            int width = outputBitmap.Width;
            int height = outputBitmap.Height;

            for (int x = 0; x < width; x++)
            {
                for (int y = 0; y < height; y++)
                {
                    // Check if the pixel is white
                    int pixelRed = outputLockBitmap.GetRedColor(x, y);
                    if (pixelRed == 255)
                    {
                        //You can turn this on if you also like to fill white spots in black areas (this will take more cpu time)
                        //outputLockBitmap.SpotFill(x, y, Color.Black, maxSpotSize);
                    }
                    else if (pixelRed == 0)
                    {
                        outputLockBitmap.SpotFill(x, y, Color.White, maxSpotSize);
                    }
                }
            }

        }

        return outputBitmap;
    }
    public static int SpotFill(this LockBitmap bitmap, int startX, int startY, Color fillColor, int maxSpotSize)
    {
        int targetColor = bitmap.GetRedColor(startX, startY);

        int pixelCount = 0;

        int width = bitmap.Width;
        int height = bitmap.Height;

        // Search for connected pixels, and save them
        int startPixel = startX * width + startY;

        Queue<int> queue = new Queue<int>(maxSpotSize);
        queue.Enqueue(startPixel);

        HashSet<int> changedPixels = new HashSet<int>();

        while (queue.Count > 0)
        {
            int pixel = queue.Dequeue();
            int x = pixel / width;
            int y = pixel % width;

            if (x < 0 || x >= width || y < 0 || y >= height)
                continue;

            if (bitmap.GetRedColor(x, y) != targetColor)
                continue;

            if (changedPixels.Contains(pixel))
                continue;

            changedPixels.Add(pixel);

            pixelCount++;

            if (pixelCount > maxSpotSize)
                return 0;

            queue.Enqueue((x + 1) * width + y);
            queue.Enqueue((x - 1) * width + y);
            queue.Enqueue(x * width + (y + 1));
            queue.Enqueue(x * width + (y - 1));
        }

        // Apply the fill
        if (pixelCount < maxSpotSize)
        {
            foreach (int pixel in changedPixels)
            {
                int x = pixel / width;
                int y = pixel % width;
                bitmap.SetPixel(x, y, fillColor);
            }
            return pixelCount;
        }

        return 0;
    }

Using the function

So you could do something like this:

Image image = Image.FromFile("C:\\Input.png");
image = image.Despeckle(maxSpotSize: 10);
image.Save("C:\\Output.png");
Sander
  • 37
  • 5
0

As found while working on a different project, instead of converting the image to a Bitmap and then using GetPixel and SetPixel functions, simply cast the Image as a bitmap to call the GetPixel and SetPixel functions. Its much more efficient and uses far less memory. Example: Instead of using img.GetPixel(ii - 1, jj - 1);, use ((Bitmap)pictureBox1.Image).GetPixel(ii-1,jj-1);

Gavin S.
  • 31
  • 5