How to remove a "green screen" portrait background

Question

I'm looking for a way to automatically remove (=make transparent) a "green screen" portrait background from a lot of pictures.

My own attempts this far have been... ehum... less successful.

I'm looking around for any hints or solutions or papers on the subject. Commercial solutions are just fine, too.

And before you comment and say that it is impossible to do this automatically: no it isn't. There actually exists a company which offers exactly this service, and if I fail to come up with a different solution we're going to use them. The problem is that they guard their algorithm with their lives, and therefore won't sell/license their software. Instead we have to FTP all pictures to them where the processing is done and then we FTP the result back home. (And no, they don't have an underpaid staff hidden away in the Philippines which handles this manually, since we're talking several thousand pictures a day...) However, this approach limits its usefulness for several reasons. So I'd really like a solution where this could be done instantly while being offline from the internet.

EDIT: My "portraits" depictures persons, which do have hair - which is a really tricky part since the green background will bleed into hair. Another tricky part is if it is possible to distingush between the green in the background and the same green in peoples clothes. The company I'm talking about above claims that they can do it by figuring out if the green area are in focus (being sharp vs blurred).

Answer 1

Since you didn't provide any image, I selected one from the web having a chroma key with different shades of green and a significant amount of noise due to JPEG compression.

There is no technology specification so I used Java and Marvin Framework.

input image:

The step 1 simply converts green pixels to transparency. Basically it uses a filtering rule in the HSV color space.

As you mentioned, the hair and some boundary pixels presents colors mixed with green. To reduce this problem, in the step 2, these pixels are filtered and balanced to reduce its green proportion.

before:

after:

Finally, in the step 3, a gradient transparency is applied to all boundary pixels. The result will be even better with high quality images.

final output:

Source code:

import static marvin.MarvinPluginCollection.*;

public class ChromaToTransparency {

    public ChromaToTransparency(){
        MarvinImage image = MarvinImageIO.loadImage("./res/person_chroma.jpg");
        MarvinImage imageOut = new MarvinImage(image.getWidth(), image.getHeight());
        // 1. Convert green to transparency
        greenToTransparency(image, imageOut);
        MarvinImageIO.saveImage(imageOut, "./res/person_chroma_out1.png");
        // 2. Reduce remaining green pixels
        reduceGreen(imageOut);
        MarvinImageIO.saveImage(imageOut, "./res/person_chroma_out2.png");
        // 3. Apply alpha to the boundary
        alphaBoundary(imageOut, 6);
        MarvinImageIO.saveImage(imageOut, "./res/person_chroma_out3.png");

    }

    private void greenToTransparency(MarvinImage imageIn, MarvinImage imageOut){
        for(int y=0; y<imageIn.getHeight(); y++){
            for(int x=0; x<imageIn.getWidth(); x++){

                int color = imageIn.getIntColor(x, y);
                int r = imageIn.getIntComponent0(x, y);
                int g = imageIn.getIntComponent1(x, y);
                int b = imageIn.getIntComponent2(x, y);

                double[] hsv = MarvinColorModelConverter.rgbToHsv(new int[]{color});

                if(hsv[0] >= 60 && hsv[0] <= 130 && hsv[1] >= 0.4 && hsv[2] >= 0.3){
                    imageOut.setIntColor(x, y, 0, 127, 127, 127);
                }
                else{
                    imageOut.setIntColor(x, y, color);
                }

            }
        }
    }

    private void reduceGreen(MarvinImage image){
        for(int y=0; y<image.getHeight(); y++){
            for(int x=0; x<image.getWidth(); x++){
                int r = image.getIntComponent0(x, y);
                int g = image.getIntComponent1(x, y);
                int b = image.getIntComponent2(x, y);
                int color = image.getIntColor(x, y);
                double[] hsv = MarvinColorModelConverter.rgbToHsv(new int[]{color});

                if(hsv[0] >= 60 && hsv[0] <= 130 && hsv[1] >= 0.15 && hsv[2] > 0.15){
                    if((r*b) !=0 && (g*g) / (r*b) >= 1.5){
                        image.setIntColor(x, y, 255, (int)(r*1.4), (int)g, (int)(b*1.4));
                    } else{
                        image.setIntColor(x, y, 255, (int)(r*1.2), g, (int)(b*1.2));
                    }
                }
            }
        }
    }

    public static void main(String[] args) {
        new ChromaToTransparency();
    }
}

Answer 2

Take a look at this thread: http://www.wizards-toolkit.org/discourse-server/viewtopic.php?f=2&t=14394&start=0

and the link within it to the tutorial at: http://tech.natemurray.com/2007/12/convert-white-to-transparent.html

Then it's just a matter of writing some scripts to look through the directory full of images. Pretty simple.

Answer 3

If you know the "green color" you may write a small program in opencv C/C++/Python to do extract that color and replace with transparent pixels.

Answer 4

123 Video Magic Green Screen Background Software and there are a few more just made to remove green screen background hope this helps

Answer 5

PaintShop Pro allows you to remove backgrounds based on picking a color. They also have a Remove Background wand that will remove whatever you touch (converting those pixels to transparent). You can tweak the "tolerance" for the wand, such that it takes out pixels that are similar to the ones you are touching. This has worked pretty well for me in the past.

To automate it, you'd program a script in PSP that does what you want and then call it from your program. This might be a kludgy way to to do automatic replacement, but it would be the cheapest, fastest solution without having to write a bunch of C#/C++ imaging code or pay a commercial agency.

They being said, you pay for what you get.