c# .NET green screen background remove

Question

I am working on a photo software for desktop PC that works on Windows 8. I would like to be able to remove the green background from the photo by means of chroma keying.

I'm a beginner in image manipulation, i found some cool links ( like http://www.quasimondo.com/archives/000615.php ), but I can't transale it in c# code.

I'm using a webcam (with aforge.net) to see a preview and take a picture. I tried color filters but the green background isn't really uniform, so this doesn't work.

How to do that properly in C#?

Answer 1

It will work, even if the background isn't uniform, you'll just need the proper strategy that is generous enough to grab all of your greenscreen without replacing anything else.

Since at least some links on your linked page are dead, I tried my own approach:

• The basics are simple: Compare the image pixel's color with some reference value or apply some other formula to determine whether it should be transparent/replaced.

• The most basic formula would involve something as simple as "determine whether green is the biggest value". While this would work with very basic scenes, it can screw you up (e.g. white or gray will be filtered as well).

I've toyed around a bit using some simple sample code. While I used Windows Forms, it should be portable without problems and I'm pretty sure you'll be able to interpret the code. Just note that this isn't necessarily the most performant way to do this.

Bitmap input = new Bitmap(@"G:\Greenbox.jpg");

Bitmap output = new Bitmap(input.Width, input.Height);

// Iterate over all piels from top to bottom...
for (int y = 0; y < output.Height; y++)
{
    // ...and from left to right
    for (int x = 0; x < output.Width; x++)
    {
        // Determine the pixel color
        Color camColor = input.GetPixel(x, y);

        // Every component (red, green, and blue) can have a value from 0 to 255, so determine the extremes
        byte max = Math.Max(Math.Max(camColor.R, camColor.G), camColor.B);
        byte min = Math.Min(Math.Min(camColor.R, camColor.G), camColor.B);

        // Should the pixel be masked/replaced?
        bool replace =
            camColor.G != min // green is not the smallest value
            && (camColor.G == max // green is the biggest value
            || max - camColor.G < 8) // or at least almost the biggest value
            && (max - min) > 96; // minimum difference between smallest/biggest value (avoid grays)

        if (replace)
            camColor = Color.Magenta;

        // Set the output pixel
        output.SetPixel(x, y, camColor);
    }
}

I've used an example image from Wikipedia and got the following result:

Just note that you might need different thresholds (8 and 96 in my code above), you might even want to use a different term to determine whether some pixel should be replaced. You can also add smoothening between frames, blending (where there's less green difference), etc. to reduce the hard edges as well.

Answer 2

I've tried Mario solution and it worked perfectly but it's a bit slow for me. I looked for a different solution and I found a project that uses a more efficient method here. Github postworthy GreenScreen

That project takes a folder and process all files, I just need an image so I did this:

private Bitmap RemoveBackground(Bitmap input)
    {
        Bitmap clone = new Bitmap(input.Width, input.Height, PixelFormat.Format32bppArgb);
        {
            using (input)
            using (Graphics gr = Graphics.FromImage(clone))
            {
                gr.DrawImage(input, new Rectangle(0, 0, clone.Width, clone.Height));
            }

            var data = clone.LockBits(new Rectangle(0, 0, clone.Width, clone.Height), ImageLockMode.ReadWrite, clone.PixelFormat);

            var bytes = Math.Abs(data.Stride) * clone.Height;
            byte[] rgba = new byte[bytes];
            System.Runtime.InteropServices.Marshal.Copy(data.Scan0, rgba, 0, bytes);

            var pixels = Enumerable.Range(0, rgba.Length / 4).Select(x => new {
                B = rgba[x * 4],
                G = rgba[(x * 4) + 1],
                R = rgba[(x * 4) + 2],
                A = rgba[(x * 4) + 3],
                MakeTransparent = new Action(() => rgba[(x * 4) + 3] = 0)
            });

            pixels
                .AsParallel()
                .ForAll(p =>
                {
                    byte max = Math.Max(Math.Max(p.R, p.G), p.B);
                    byte min = Math.Min(Math.Min(p.R, p.G), p.B);

                    if (p.G != min && (p.G == max || max - p.G < 7) && (max - min) > 20)
                        p.MakeTransparent();
                });

            System.Runtime.InteropServices.Marshal.Copy(rgba, 0, data.Scan0, bytes);
            clone.UnlockBits(data);

            return clone;
        }
    }

Do not forget to dispose of your Input Bitmap and the return of this method. If you need to save the image just use the Save instruction of Bitmap.

clone.Save(@"C:\your\folder\path", ImageFormat.Png);

Here you can find methods to process an image even faster.Fast Image Processing in C#

Answer 3

Chromakey on a photo should assume an analog input. In the real world, exact values are very rare.

How do you compensate for this? Provide a threshold around the green of your choice in both hue and tone. Any colour within this threshold (inclusive) should be replaced by your chosen background; transparent may be best. In the first link, the Mask In and Mask Out parameters achieve this. The pre and post blur parameters attempt to make the background more uniform to reduce encoding noise side effects so that you can use a narrower (preferred) threshold.

For performance, you may want to write a pixel shader to zap the 'green' to transparent but that is a consideration for after you get it working.