10

I create an image with PIL:

example image

I need to fill in the empty space (depicted as black). I could easily fill it with a static color, but what I'd like to do is fill the pixels in with nearby colors. For example, the first pixel after the border might be a Gaussian blur of the filled-in pixels. Or perhaps a push-pull type algorithm described in The Lumigraph, Gortler, et al..

I need something that is not too slow because I have to run this on many images. I have access to other libraries, like numpy, and you can assume that I know the borders or a mask of the outside region or inside region. Any suggestions on how to approach this?

UPDATE:

As suggested by belisarius, opencv's inpaint method is perfect for this. Here's some python code that uses opencv to achieve what I wanted:

import Image, ImageDraw, cv

im = Image.open("u7XVL.png")
pix = im.load()

#create a mask of the background colors
# this is slow, but easy for example purposes
mask = Image.new('L', im.size)
maskdraw = ImageDraw.Draw(mask)
for x in range(im.size[0]):
    for y in range(im.size[1]):
        if pix[(x,y)] == (0,0,0):
            maskdraw.point((x,y), 255)

#convert image and mask to opencv format
cv_im = cv.CreateImageHeader(im.size, cv.IPL_DEPTH_8U, 3)
cv.SetData(cv_im, im.tostring())
cv_mask = cv.CreateImageHeader(mask.size, cv.IPL_DEPTH_8U, 1)
cv.SetData(cv_mask, mask.tostring())

#do the inpainting
cv_painted_im = cv.CloneImage(cv_im)
cv.Inpaint(cv_im, cv_mask, cv_painted_im, 3, cv.CV_INPAINT_NS)

#convert back to PIL
painted_im = Image.fromstring("RGB", cv.GetSize(cv_painted_im), cv_painted_im.tostring())
painted_im.show()

And the resulting image:

painted image

jterrace
  • 64,866
  • 22
  • 157
  • 202

3 Answers3

8

A method with nice results is the Navier-Stokes Image Restoration. I know OpenCV has it, don't know about PIL.

Your example:

enter image description here enter image description here

I did it with Mathematica.

Edit

As per your reuquest, the code is:

i = Import["https://i.stack.imgur.com/uEPqc.png"];
Inpaint[i, ColorNegate@Binarize@i, Method -> "NavierStokes"]

The ColorNegate@ ... part creates the replacement mask. The filling is done with just the Inpaint[] command.

Dr. belisarius
  • 60,527
  • 15
  • 115
  • 190
  • Awesome. I've put some sample python code that converts from PIL to opencv and does the painting in my question. Accepted your answer. Thanks! – jterrace Aug 18 '11 at 14:28
  • @jterrace Glad to help! I voted up your question too, because it is a very good example of how useful SO can be. – Dr. belisarius Aug 18 '11 at 19:59
  • any idea why inpaint always misses x=0 and y=0 (the top row and left column) of the image? – jterrace Oct 04 '11 at 22:55
0

Depending on how you're deploying this application, another option might be to use the Gimp's python interface to do the image manipulation.

The doc page I linked to is oriented more towards writing GIMP plugins in python, rather than interacting with a background gimp instance from a python app, but I'm pretty sure that's also possible (it's been a while since I played with the gimp/python interface, I'm a little hazy).

Eli Collins
  • 8,375
  • 2
  • 34
  • 38
  • Does it have a routine to do something like this? – jterrace Aug 17 '11 at 19:52
  • Sadly, I'm not sure if it has a plugin that matches what you want... I mainly posted it as an option you might not be aware of, since it has have some more powerful tools to build from: masks, gaussian blur filters, and a pythonic interface to the lowlevel pixel region objects. Now that I think about it more, it might be bloated overkill for your purposes, especially if you have to process a lot of files. Though one algorithm that might make it worth it is the [resynthesizer](http://www.logarithmic.net/pfh/resynthesizer) plugin for the Gimp, which is a "content aware fill" algorithm. – Eli Collins Aug 17 '11 at 22:56
-1

You can also create the mask with the function CreateImage(), for instance:

inpaint_mask = cv.CreateImage(cv.GetSize(im), 8, 1)
Ben
  • 51,770
  • 36
  • 127
  • 149
RogerG
  • 1