Automatic face detection using Picasa API to extract individual images

Question

(A similar question has been asked on superuser for answers related to applications. The question is posted here to gather programmable solutions for the same)

At my work place, passport sized photographs are scanned together, then cut up into individual pictures and saved with unique file numbers. Currently we use Paint.net to manually select, cut and save the pictures.

Sample Scanned Document Picasa Screenshot: (from: google image search multiple sources, fairuse)

For eg. In Picasa 3.8, On clicking View > People, all the faces are shown and I am asked to name them, can I save these individual pictures automatically with the names as different pictures?

Updated

All I want to do is convert the picture above to individual pictures.

In the image above, I have shown how Picasa 3.8 detects the images and prompts me to name them. I do not need face recognition, I simply need face detection. Picasa detects the individual images and shows them on the RHS. These individual images are what I need. Picasa creates a .ini file which saves the hex values which contains the co-ordinates of the individual faces.

These individual faces are what I am interested in If I can have the co-ordinates, I can crop the required images from the picture.

SAMPLE.jpg

ini contents

 [SAMPLE.jpg]
faces=rect64(c18f4c8ef407851e),d4ff0a020be5c3c0;rect64(534a06d429ae627),dff6163dfd9d4e41;rect64(b9c100fae46b3046),e1059dcf6672a2b3;rect64(7b5105daac3a3cf4),4fc7332c107ffafc;rect64(42a036a27062a6c),ef86c3326c143248;rect64(31f4efe3bd68fd8),90158b3d3b65dc9b;rect64(327904e0614d390d),43cbda6e92fcb63e;rect64(4215507584ae9b8c),15b6a967e857f334;rect64(895d4efeb8b68425),5c4ff70ac70b27d3
backuphash=3660

*The ini file seems to be saving the co-ordinates of the face tags as rect64(534a06d429ae627),dff6163dfd9d4e41 for each tag. Quoting from Picasa Help Site user Technonath says

@oedious wrote:- This is going to be somewhat technical, so hang on. * The number encased in rect64() is a 64-bit hexadecimal number. * Break that up into four 16-bit numbers. * Divide each by the maximum unsigned 16-bit number (65535) and you'll have four numbers between 0 and 1. * The four numbers remaining give you relative coordinates for the face rectangle: (left, top, right, bottom). * If you want to end up with absolute coordinates, multiple the left and right by the image width and the top and bottom by the image height.

The above quote talks about the number encased in rect64() what about the number outside the parentheses after the comma?

I have asked a related question. Answers of which may help you too. Get four 16bit numbers from a 64bit hex value

Note: The ini details are the same which picasa generated for the particular image.

Plus the question has been updated multiple times and may not be clear enough.

There are some responses at the Picasa Help site, where I asked the same question One of the answers from that thread to get co-ordinates based on the hex values from the ini file. The following code is in C# from esac from the help site. Can I do the same in PHP?

public static RectangleF GetRectangle(string hashstr)
{
    UInt64 hash = UInt64.Parse(hashstr, System.Globalization.NumberStyles.HexNumber);
    byte[] bytes = BitConverter.GetBytes(hash);

    UInt16 l16 = BitConverter.ToUInt16(bytes, 6);
    UInt16 t16 = BitConverter.ToUInt16(bytes, 4);
    UInt16 r16 = BitConverter.ToUInt16(bytes, 2);
    UInt16 b16 = BitConverter.ToUInt16(bytes, 0);

    float left = l16 / 65535.0F;
    float top = t16 / 65535.0F;
    float right = r16 / 65535.0F;
    float bottom = b16 / 65535.0F;

    return new RectangleF(left, top, right - left, bottom - top);
} 

PHP code trying to convert 64bit to numbers between 1 and 0

<?php
$dim = getimagesize("img.jpg");    
$hex64=array();
$b0="c18f4c8ef407851e";
$hex64[]=substr($b0,0,4);
$hex64[]=substr($b0,4,4);
$hex64[]=substr($b0,8,4);
$hex64[]=substr($b0,12,4);
$width=$dim[0];
$height=$dim[1];
foreach($hex64 as $hex16){
$dec=hexdec($hex16);
$divide=65536;
$mod=$dec%$divide;
$result=$dec/$divide;
$cordinate1=$result*$width;
$cordinate2=$result*$height;
echo "Remainder 1 : ".$mod." ; Result 1 :  ".$result."<br/>CO-ORDINATES : <B>".$cordinate1." ".$cordinate2."</B><br/>";
}
?>

The output

Remainder 1 : 49551 ; Result 1 : 0.75608825683594 CO-ORDINATES : 371.99542236328 396.94633483887 Remainder 1 : 19598 ; Result 1 : 0.29904174804688 CO-ORDINATES : 147.12854003906 156.99691772461 Remainder 1 : 62471 ; Result 1 : 0.95323181152344 CO-ORDINATES : 468.99005126953 500.4467010498 Remainder 1 : 34078 ; Result 1 : 0.51998901367188 CO-ORDINATES : 255.83459472656 272.99423217773

So I have the co-ordinates too and @Nirmal has shown how to crop them. Now next steps would be to parse picasa.ini for the hex codes and file names and integrate the code. Picasa doesn't currently provide the hex codes via a api(or Do they?). If that were the case, things would have been better.

So we are nearing a solution. Thank you all, I wish I could award the bounty to everyone(I cannot, but fear not and look out for a spike in your rep!)

Answer 1

Look at OpenCV - one of the examples that comes with the distribution is for face detection.

Answer 2

Your solution to the problem is overkill. Ignore the faces. What you have is a solid white background and a bunch of rectangular images on it. All you need to do is find the rectangle that encloses each image and crop.

Start by running a filter over the original image that marks all non-background pixels. This will take some tuning because sometimes the background will have a touch of tint in it (dirt) or the photo will have some pixels that look like the background (really white teeth).

Now you look for large areas with no background color in them. Crop those into rectangles.

Since you are the one doing the scanning, why not make the background green? Green might be an easier color to filter, especially since the passport photos are taken on a white background.

Answer 3

To answer the picasa question, see this response on the picasa forums:
http://www.google.com/support/forum/p/Picasa/thread?tid=36ae553a7b49088e&hl=en

@oedious wrote:- This is going to be somewhat technical, so hang on. * The number encased in rect64() is a 64-bit hexadecimal number. * Break that up into four 16-bit numbers. * Divide each by the maximum unsigned 16-bit number (65535) and you'll have four numbers between 0 and 1. * The four numbers remaining give you relative coordinates for the face rectangle: (left, top, right, bottom). * If you want to end up with absolute coordinates, multiple the left and right by the image width and the top and bottom by the image height.

Answer 4

You can simplify the problem even further :-) if the scanned images will always be in a 5x4 grid ... then you can easily just open the image in just about any programming language that offers bitmap manipulation, and save each square. Here's an example of how to do this with C#:

private Image Crop(Image pics, Rectangle area)
{
   var bitmap = new Bitmap(pics);
   return (Image)bitmap.Clone(area, bitmap.PixelFormat);
}

All you'd need to do is calculate each rectangle, and then call this method which returns just the area of the image defined by the rectangle. Something like (possibly pseudo code, haven't compiled the code below):

// assuming that each sub image in the larger is 45x65
int cellwidth=45, cellheight=65;

for(int row=0;row<5;row++)
{
  for(int col=0;col<4;col++)
  {
    var rect = new Rectangle(
      row * cellwidth,
      col * cellheight,
      cellwidth,
      cellheight);
    var picture = Crop(bigPicture, rect);
    // then save the sub image with whatever naming convention you need
  }
}

Answer 5

For the cropping part, I am typing the code without testing, but this should work:

<?php
//source image
$srcImg = "full/path/of/source/image.jpg";
//output image
$outImg = "full/path/to/result/image.jpg";

//coordinates obtained from your calculation
$p1 = array('X'=>371, 'Y'=>156);
$p2 = array('X'=>468, 'Y'=>156);
$p3 = array('X'=>468, 'Y'=>272);
$p4 = array('X'=>371, 'Y'=>272);

//let's calculate the parametres
$srcX = $p1['X'];
$srcY = $p1['Y'];
$width = $p2['X'] - $p1['X'];
$height = $p4['Y'] - $p1['Y'];

//image processing
$srcImg = imagecreatefromjpeg($srcImg);
$dstImg = imagecreatetruecolor($width, $height);
imagecopy($dstImg, $srcImg, 0, 0, $srcX, $srcY, $width, $height);
imagejpeg($dstImg, $outImg, 100); // 100 for highest quality, 0 for lowest quality
imagedestroy($dstImg);
?>

The above code assumes that your source image is in JPEG format and the coordinates make a perfect rectangle or square.

Hope that helps.

Answer 6

This should get you across the finish line. Here's some code to parse the INI.

<?php
$vals = parseIni('picasa.ini');
foreach($vals as $filename => $values) {
    $rects = getRects($values['faces']);
    foreach($rects as $rect) {
        printImageInfo($filename, $rect);
    }
}

/**
 * PHP's own parse_ini_file doesn't like the Picasa format.
 */
function parseIni($file)
{
    $index = 0;
    $vals = array();
    $f = fopen($file, 'r');
    while(!feof($f)) {
        $line = trim(fgets($f));
        if (preg_match('/^\[(.*?)\]$/', $line, $matches)) {
            $index = $matches[1];
            continue;
        }

        $parts = explode('=', $line, 2);
        if (count($parts) < 2) continue;
        $vals[$index][$parts[0]] = $parts[1];
    }

    fclose($f);
    return $vals;
}

function getRects($values)
{
    $values = explode(';', $values);
    $rects = array();
    foreach($values as $rect) {
        if (preg_match('/^rect64\(([^)]+)\)/', $rect, $matches)) {
            $rects[] = $matches[1];
        }
    }

    return $rects;
}

function printImageInfo($filename, $rect)
{
    $dim = getimagesize($filename);    
    $hex64=array();
    $hex64[]=substr($rect,0,4);
    $hex64[]=substr($rect,4,4);
    $hex64[]=substr($rect,8,4);
    $hex64[]=substr($rect,12,4);
    $width=$dim[0];
    $height=$dim[1];
    foreach($hex64 as $hex16){
        $dec=hexdec($hex16);
        $divide=65536;
        $mod=$dec%$divide;
        $result=$dec/$divide;
        $cordinate1=$result*$width;
        $cordinate2=$result*$height;
        echo "Remainder 1 : ".$mod." ; Result 1 :  ".$result."<br/>CO-ORDINATES : <B>".$cordinate1." ".$cordinate2."</B><br/>";
    }
}

Answer 7

I've developed a little app in .NET that does exactly what you said, it produces the files for the faces. Check it out here: http://ceottaki.com/devprojects/getpicasafaces

The source code is available as well.

While I haven't implemented getting the name of the contacts from their hexadecimal code, it is possible using the Google Contacts API: http://code.google.com/apis/contacts/

With that API it is possible to get contacts by ID, and if your contacts are synced between Picasa and Google Contacts, the hexadecimal ID is the same.

The last part of a full contact link is the hexadecimal used by Picasa.

I hope this helps.

Cheers, Felipe.