How can I make my JavaScript programs run faster?

Question

Ok, so I am working on a sort of detection system where I will be pointing the camera at a screen, and it will have to find the red object. I can successfully do this, with pictures, but the problem is that it takes several seconds to load. I want to be able to do this to live videos, so I need it to find the object immediately. Here is my code:

video.addEventListener('pause', function () {

let reds = [];

for(x=0; x<= canvas.width; x++){
for(y=0; y<= canvas.height; y++){

let data = ctx.getImageData(x, y, 1, 1).data;
let rgb = [ data[0], data[1], data[2] ];

if (rgb[0] >= rgb[1] && rgb[0] >=rgb[2] && !(rgb[0]>100 && rgb[1]>100 && rgb[2]>100) && rgb[1]<100 && rgb[2]<100 && rgb[0]>150){
reds[reds.length] = [x, y]
}

let addedx = 0
let addedy = 0

for(i=0; i<reds.length; i++){
    addedx = addedx + reds[i][0]
    addedy = addedy + reds[i][1]
}

let center = [addedx/reds.length, addedy/reds.length]

ctx.rect(center[0]-5, center[1]-5, 10, 10)
ctx.stroke() 

}, 0);

Ya, I know its messy. Is there something about the for loops that are slow? I know I'm looping through thousands of pixels but that's the only way I can think of to do it.

Answer 1

I would run the detection algorithm in a webassembly module. Since it is just pixel data, thats right up its alley.

You could then pass individual frames to a different instance of the wasm module.

As far as answering your question directly, I would grab the whole frame, not 1 pixel at a time, or you might get pixels sampled from different frames. You can then submit that frame to a worker, you could even divide up the frame and send them to different workers (or as previously mentioned a wasm module)

Also since you have an array you can use Arrray.map and Array.reduce to get you to just the red values, and how big they are by testing for adjacent pixels, instead of all the comparison. Not sure if it will be faster but worth a try.

Answer 2

For the speed, you should consider all your process:

• more your language is near the machine language, better your result will be. Saying so, C++ is better for the algorithm.
• CPU speed is your friend. Launching your code on an Atom processor or on an i7 processor, is like night and day. Moreover, some type of processor is dedicated for vision like VPU

For your code:

• You try to rewrite code that already exists. You can find good examples of detection in the great OpenCV library: https://www.learnopencv.com/invisibility-cloak-using-color-detection-and-segmentation-with-opencv

Hope it help you :)

Answer 3

As it has been said, Javascript is not the most performant for this task. However, here are some things I noticed, which could slow you down.

1. You grab the image data one pixel at a time. Since this method can return the whole frame, you can do this once.

2. Optimize your isRed condition:

rgb[0] >= rgb[1] &&                                // \
rgb[0] >= rgb[2] &&                                //  >-- This is useless
!(rgb[0] > 100 && rgb[1] > 100 && rgb[2] > 100) && // /
rgb[1] < 100 && // \
rgb[2] < 100 && //  >-- These 3 conditions imply the others
rgb[0] > 150    // /

3. You calculate the center inside your for loop after each pixel, but it would only make sense after processing the whole frame.

4. Since the video feed is coming from a camera, maybe you don't need to look at every single pixel. Maybe every 5 pixels is enough? That's what the example below does. Tweak this.

Demo including these optimizations

Node: This demo includes an adaptation of the code from this answer, to copy the video onto the canvas.

const video  = document.getElementById("video"),
      canvas = document.getElementById("canvas"),
      ctx    = canvas.getContext("2d");

let width,
    height;

// To make this demo work
video.crossOrigin = "Anonymous";

// Set canvas to video size when known
video.addEventListener("loadedmetadata", function() {
  width = canvas.width = video.videoWidth;
  height = canvas.height = video.videoHeight;
});

video.addEventListener("play", function() {
  const $this = this; // Cache

  (function loop() {
    if (!$this.paused && !$this.ended) {
      ctx.drawImage($this, 0, 0);
      
      const reds = [],
            data = ctx.getImageData(0, 0, width, height).data,
            len  = data.length;

      for (let i = 0; i < len; i += 5 * 4) { // 4 because data is made of RGBA values
        const rgb = data.slice(i, i + 3);

        if (rgb[0] > 150 && rgb[1] < 100 && rgb[2] < 100) {
          reds.push([i / 4 % width, Math.floor(i / 4 / width)]); // Get [x,y] from i
        }
      }

      if (reds.length) { // Can't divide by 0
        const sums = reds.reduce(function (res, point) {
          return [res[0] + point[0], res[1] + point[1]];
        }, [0,0]);

        const center = [
          Math.round(sums[0] / reds.length),
          Math.round(sums[1] / reds.length)
        ];

        ctx.strokeStyle = "blue";
        ctx.lineWidth = 10;
        ctx.beginPath();
        ctx.rect(center[0] - 5, center[1] - 5, 10, 10);
        ctx.stroke();
      }

      setTimeout(loop, 1000 / 30); // Drawing at 30fps
    }
  })();
}, 0);

video, canvas { width: 250px; height: 180px; background: #eee; }

<video id="video" src="https://shrt-statics.s3.eu-west-3.amazonaws.com/redball.mp4" controls></video>
<canvas id="canvas"></canvas>