Fastest way to compute & feed an arbitrary color centroid in an image to PHP

Question

I'm looking for the fastest way to compute a directional vector based on an arbitrary color in an image (a Rpi camera, but a JPEG file for testing is OK for now), a.k.a. tracking a colored ball project. Please note that the resulting vector (or centroid coordinates, whatever) needs to be passed to PHP for the program execution, so the solution I'm looking for needs to end with PHP, but can be anything before, given it can be implemented on both Windows and Linux.

Consider an input JPEG image:

Here are 2 example directional vectors I'm after, obtained based on a 1) teal color input and 2) purple color input. Obviously, only 1 vector will ever be asked at a time, I put 2 to demonstrate multiple examples into 1 image, but it's always gonna be only 1 vector at a time. Note that the resulting vectors ("v") are standardized to -1.0 (bottom/left) to +1.0 (bottom/right) so that zero is the middle of the picture.

Here are the various solutions I've implemented/tested so far and how much time the whole process takes, based on a 960x640 JPEG picture, but the implemented solution will be tied to a Rpi camera input, I do not have the camera yet so I use a JPEG image until the camera arrives from China.

1) 2700ms : Use GD2 that is bundled with PHP, for loop over each pixels, push pixels matching ~10% RGB values in XY arrays, average the XY arrays, compute/normalize directional vector from XY arrays.

$arr_matching_pixels = array('arr_x' => array(), 'arr_y' => array());
for($y = 0; $y < $h - 1; $y++){
    for($x = 0; $x < $w - 1; $x++){
        $arr_pixel = imagecolorsforindex($img, imagecolorat($img, $x, $y));
        if(abs($arr_pixel['red'] - $arr_seek_color['red']) < 30){
            if(abs($arr_pixel['green'] - $arr_seek_color['green']) < 30){
                if(abs($arr_pixel['blue'] - $arr_seek_color['blue']) < 30){
                    array_push($arr_matching_pixels['arr_x'], $x);
                    array_push($arr_matching_pixels['arr_y'], $y);
                }
            }
        }
    }
}
// Compute centroid of color... etc...

2) 700ms : Same as #1 except begin by resizing the canvas by 50% (acceptable loss) using imagecreatefromjpeg('_test_cam_img.jpg');

3) 560ms : Same as #2 except use ImageMagick with a pixel iterator loop to read the pixels

$imagick = new Imagick(realpath($o_img));
$arr_matching_pixels = array('arr_x' => array(), 'arr_y' => array());
$arr_pixel = array();
$iterator = $imagick->getPixelIterator();
foreach($iterator as $y => $pixels){
    foreach($pixels as $x => $pixel){
        $arr_pixel = $pixel->getColor();
        if(abs($arr_pixel['r'] - $arr_seek_color['red']) < 30){
            if(abs($arr_pixel['g'] - $arr_seek_color['green']) < 30){
                if(abs($arr_pixel['b'] - $arr_seek_color['blue']) < 30){
                    array_push($arr_matching_pixels['arr_x'], $x);
                    array_push($arr_matching_pixels['arr_y'], $y);
                }
            }
        }
    }
}
// Compute centroid of color... etc...

4) 340ms : Call the system's ImageMagick binary via the exec() function, pass it the image location, the chroma/color key, a resize by 50% param, a 10% fuzz param, and the sparse-color: modifier to extract a textual (CSV-like) list representation of desired pixels, then use PHP to loop over each line, explode commas and push all pixels in XY arrays, average the XY arrays, compute/normalize directional vector from XY arrays. I noted that calling exec() proves to be quite slower than executing the same command directly from the Windows command line.

$imagick = new Imagick(realpath($o_img));
$out = exec('"E:\Users\Ben\Roaming Apps\imagemagick-6.9.3\convert" E:\wamp64\www\test_cam_img.jpg -resize 50% -fuzz 10% +transparent rgb(' . $arr_seek_color['red'] . ',' . $arr_seek_color['green'] . ',' . $arr_seek_color['blue'] . ') sparse-color:');
$arr_lines = explode(' ', $out);
$arr_matching_pixels = array('arr_x' => array(), 'arr_y' => array());
foreach($arr_lines as $str_line){
    $arr_xy_coords = explode(',', $str_line);
    array_push($arr_matching_pixels['arr_x'], $arr_xy_coords[0]);
    array_push($arr_matching_pixels['arr_y'], $arr_xy_coords[1]);
}
// Compute centroid of color... etc...

5) 32ms : PHP creates an "in" text file containing the image path and the chroma/color key and begins looping until it reads an "out" text file. A python+OpenCV script already/always runs a (stoppable) infinite loop constantly looking for an "in" text file and when it exists, it read it, explodes the values, makes a 1-bit mask using the HSV values ~10% (cv2.inRange) from the "in" file, then makes an array using cv2.findNonZero(mask) and computes the array mean value and writes it to an "out" text file that PHP immediately reads, containing the directional vector value. This is by far, the fastest way I have found, but it is awkward, because it implies that the python script will have to be programmed in a CRONJOB and monitored/relaunched in a single instance if it crashes.

file_put_contents('_avg_color_coords_in.txt', $o_img . "\n" . $arr_seek_color['h'] . ',' . $arr_seek_color['s'] . ',' . $arr_seek_color['l']);

$starttime = time();
while((time() - $starttime) < 5){ // Max 5 seconds (exaggerated)
    if(file_exists('_avg_color_coords_out.txt')){
        $dir_vector = (float) file_get_contents('_avg_color_coords_out.txt');
        if(!@unlink('_avg_color_coords_out.txt')){
            sleep(1);
            unlink('_avg_color_coords_out.txt');
        }
        break;
    }
    usleep(2000);
}
// $dir_vector ("v", the centroid of the color) is already computed by Python


// ---------- PYTHON SCRIPT ----------
import math
import cv2
import numpy as np
import os
import time

#cap = cv2.VideoCapture(0)

#while (1):
#    _, frame = cap.read()
if(os.path.exists('_avg_color_coords_stop.txt')):
    exit()
while not os.path.exists('_avg_color_coords_in.txt'):
    time.sleep(0.002)
f = open('_avg_color_coords_in.txt', 'r')
imgsrc = f.readline().rstrip('\n')
rgbcol = [int(x) for x in f.readline().rstrip('\n').split(',')]
frame = cv2.imread(imgsrc)
h, w = frame.shape[:2]

hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
hfacl = rgbcol[0] / 360 * 180 * 0.95
hfach = rgbcol[0] / 360 * 180 * 1.05
sfacl = rgbcol[1] / 100 * 255 * 0.9
sfach = rgbcol[1] / 100 * 255 * 1.1
vfacl = rgbcol[2] / 100 * 255 * 0.9
vfach = rgbcol[2] / 100 * 255 * 1.1
lower_color = np.array([hfacl, sfacl, vfacl]) # 0..180, 0..255, 0..255 not percentage!
upper_color = np.array([hfach, sfach, vfach]) # 0..180, 0..255, 0..255 not percentage!
mask = cv2.inRange(hsv, lower_color, upper_color)
#cv2.imshow('mask', mask)

points = cv2.findNonZero(mask)
if(points.any()):
    avg = np.mean(points, axis=0)
else:
    avg = [0,0]
#print(avg)

v = -math.atan(((w * 0.5) - avg[0][0]) / (h - avg[0][1])) / (3.1415 * 0.5);
f2 = open('_avg_color_coords_out.txt', 'w+')
f2.write("%s" % str(v))

#    k = cv2.waitKey(5) & 0xff
#    if k == 27:
#        break

#cv2.destroyAllWindows()
#cap.release()

f2.close()
f.close()
os.remove('_avg_color_coords_in.txt')

6) 38ms : Same as #5 except begin by resizing the canvas by 50% (acceptable loss) which doesn't seem to speed up things at all, and even seems counterproductive a little bit.

Is there a faster way or is this optimal? This will run every second on a 900mhz Rpi, so it needs to be quick. I think 30ms on a 900mhz CPU will be around 150-200ms (not tested yet, waiting for the camera to ship)

Answer 1

I had a quick go in php-vips:

#!/usr/bin/env php
<?php

require __DIR__ . '/vendor/autoload.php';

use Jcupitt\Vips;

$image = Vips\Image::newFromFile($argv[1], ['access' => 'sequential']);

# Target colour in RGB.
$target = [50, 10, 100];

# Select pixels where all bands are less than 10 away from the target.
# (and render it to memory ... we'll be reusing this mask image).
# The mask image will have one band with 0 for false and 255 for true.
$mask = $image->subtract($target)->abs()->less(10)->bandand()->copyMemory();

# The number of set pixels in the mask.
$n_set = $mask->avg() * $mask->width * $mask->height / 255;

# Handy for debugging: uncomment to write the mask image for inspection.
# $mask->writeToFile("x.png");

# Make a two-band image where band 0 is x coordinates and band 1 is y
# coordinates.
$coords = Vips\Image::xyz($mask->width, $mask->height);

# Make an indexed histogram: sum $coords at each position.
$pos = $coords->hist_find_indexed($mask);

# fetch the sum of the 255 value (true) pixels
[$x_sum, $y_sum] = $pos->getpoint(255, 0);

echo("x = " . $x_sum / $n_set . "\n");
echo("y = " . $y_sum / $n_set . "\n");

I can run it like this:

$ time ./locate-rgb.php ~/pics/x.jpg
x = 483.375
y = 487.75
real    0m0.079s
user    0m0.085s
sys 0m0.022s

So about 80ms on this modest laptop. That includes PHP startup and shutdown, and decompressing the JPG image.

That's only going to work in very constrained lighting and camera setups, but perhaps that's OK? It would be easy to make the ball detection fancier, but of course it would slow it down a bit.