Track eye balls using tracking.js

Question

I am working with a red eye remover effect with html5 canvas. I used tracking.js to track the eyes. Check the following code.

Html

<!-- Include tracking js libraries -->
<script src="js/tracking-min.js"></script>
<script src="js/eye-min.js"></script>

<img id="sourceImg" src="red_eye1.jpg"  width="202" height="187">
<canvas id="myCanvas" width="202" height="187" style="border:1px solid #d3d3d3;"></canvas>

Javascript

document.getElementById("sourceImg").onload = function() {
    var c = document.getElementById("myCanvas");
    var ctx = c.getContext("2d");
    var img = document.getElementById("sourceImg");
    ctx.drawImage(img, 0, 0);

    // track eyes
    var tracker = new tracking.ObjectTracker(['eye']);
    tracker.setStepSize(1.7);
    tracking.track('#sourceImg', tracker);
    tracker.on('track', function(event) {
        event.data.forEach(function(rect) {
            // rect consists area of each eyes.
            var imgData = ctx.getImageData(rect.x, rect.y, rect.width, rect.height);
            // remove red eye
            var i;
            for (i = 0; i < imgData.data.length; i += 4) {
                // find and check red intensity
                var redIntensity = (imgData.data[i] / ((imgData.data[i+1] + imgData.data[i+2]) / 2));
                if (redIntensity > 1.5)  // 1.5 because it gives the best results
                {
                    // reduce red to the average of blue and green
                    imgData.data[i] = (imgData.data[i+1] + imgData.data[i+2]) / 2;
                }
            }
            ctx.putImageData(imgData, rect.x, rect.y);

        });
    });  
};

I got the following result. enter image description here

Here the red eye removed successfully. But it affected the color around the eye area. Because the tracking.js returned the eye area including the surrounding area. How to track the eye balls, which will solve this issue.

markE · Answer 1 · 2015-03-10T19:54:57.150

A quick look at TrackingJS shows that it doesn't offer to take your red-bordered "eye region" and refine it to just the Iris.

So you will have to find a tracking app that does offer this refinement or code the refinement yourself.

Recommending an iris tracking library is off-topic in Stackoverflow and a code answer which does iris tracking is too broad to qualify as a good Stackoverflow answer.

Having said that, here is a verrrrrry broad outline for finding the Iris:

Start with the "eye region" detected by TrackingJS,
Get the bounding box of the sclera by color testing for the whitest pixels in that eye region,
In this bounding box get the circular pupil area by color testing for the darkest pixels between the 2 sclera regions.
Using the circular pupil as a basis, find the iris by expanding that pupil circle's radius outward until you reach the white sclera.

[ Additional thoughts ]

I continued thinking about the many nonstandard situations that need to be handled:

The eyelid is closed or squinting,
The iris is fully left or right leaving no whitish boundary encapsulating the iris,
The sclera is not colored distinctly enough due to being jaundiced or bloodshot,
Eye cosmetics that might be whiter than the sclara,
Photo imperfections like lens flare and reflections,
And more I'm not thinking of!

If your project allows, it would be much easier if the user clicks on the 2 pupils and expand the pupil radius to define the iris. Pupils are likely to be both darker than other eye structures and will be round (but possibly clipped by the eyelids).

If your project can't allow the user to select the pupils, you might use an "edge detection" algorithm to isolate the pupil (maybe Sobel Edge detection).

From experience: I can tell that this will likely not work as well as it sounds like. No two images are identical in colors (white balance, light temperature, soft/hard light, shadow, skin color, ambient light, incident light and so on) and in addition you will have factors such as noise, color of eye ball etc. Using color will force you to tweak the parameters to fit this one image. It will probably not work for the next image.. Instead of using colors you need to go with feature detections. My 2 cents.. — , Mar 10 '15 at 11:33
@KenFyrstenberg, Can you suggest any library for finding iris? — Sajith, Mar 10 '15 at 11:55
@Sajith I would think tracker.js could be used to first do eye detection, then feature detection within that region (it seem to also support custom tracking), but I am not familiar with the library. Don't know of any other unfortunately. — , Mar 10 '15 at 13:56
@KenFyrstenberg. Yes, I wrote this post just before going to bed but my mind kept thinking about the many non-standard cases (as you commented) that need to be handled :-( I've added these worries to my post. Still, using color variations still seems to me to be a good "first pass" to catch the most common eye situations (==eye open with few occluding conditions) But, perhaps your idea of using feature detection (maybe Sobel edge detection) would be more direct if a large quantity of occluding conditions likely apply). It's an interesting question that I wish I had more time to explore! — markE, Mar 10 '15 at 20:03

Track eye balls using tracking.js

1 Answers1