0

I'm displaying a live video stream in a HTML5 canvas which works fine.

Now, what I need to do is to check for any "motion" in the camera which is being displayed in the HTML5 Canvas.

During my research, I found out that this can be done by checking the previous frame with the current frame that's being displayed in the canvas.

So I tried this code within a setInterval function:

  var c = document.querySelector('.mycanv');
  var ctx = c.getContext("2d");

  var imageData = ctx.getImageData(0, 0, 200, 200);
  var data = imageData.data.length;

  console.log(data);

However, when I look in the console, the Number that the variable data outputs is always the same which is 16000! and it wont change even if there is a movemnet infront of the camera.

I'm not entirely sure if I am on a right track.

Could someone please advice on this issue and point me in a right direction please?

Thanks in advance.

Here is a minified and simple example:

https://jsfiddle.net/2648xwgz/

Basically, the code draws a frame of the video in the canvas.

Now, what I need to do is to check if the previous frame/image in the canvas is the same as the current frame/image.

Second EDIT:

Okay, so I've taken all the advice in the comments on board and tried to come up with something that actually doesn't check for all the random pixels in the images as that is heavy and not a good practic...

So, I tried something like this:

https://jsfiddle.net/qpxjcv3a/6/

The above code will run fine in Firefox and with a local Video file. but on JSFIDDLE, you will get a cross origion error.

Anyway, The key point in the code above is using the ctx.globalCompositeOperation = 'difference'; i guess.

And then doing a "scrore" calculation taken from here to detect some changes.

However, when I run my code, I always get: console.log('we have motion'); in the console! Even when the video is paused and there is no more new frames.

So I did console.log(imageScore); and it is always adding up by 10000 even when the video is paused or ended! I'm not sure why that is and whether this calculation is correct at all. But that is where I am at the moment.

Any pointers and help appreciated.

william wolly
  • 315
  • 1
  • 5
  • 16
  • 2
    The `length` of the imageData is the same, because they contain the same number of pixels. You need to compare the pixel data itself (possibly using [something like this](https://stackoverflow.com/questions/11754982/comparing-2-imagedata-objects), though there may be more current techniques than what's shown in that older question) – Daniel Beck Sep 08 '18 at 13:16
  • @DanielBeck, that makes sense. However, the link you posted is referring to two images. I know that i am trying to get the frames from the canavas which essentially is classed as different images but the images are removed before the next one is added. So, I'm not entirely sur ehow that could help this situation if i'm honest. – william wolly Sep 08 '18 at 13:40
  • Sorry it won't help you much, but just checking if two frames are the same won't tell you if there are any motion. Every frame will have noise and this noise is made of randomness which by definition will be different in every frame. So yes, you can detect this frame change by iterating through all the pixels data available in the two ImageData.data, but once again, this is not enough to detect motion... – Kaiido Sep 08 '18 at 13:57
  • @Kaiido what I said is based on what others have said and done: Examples: https://codersblock.com/blog/motion-detection-with-javascript/ and https://hackernoon.com/motion-detection-in-javascript-2614adea9325 and https://benjaminhorn.io/code/motion-detection-with-javascript-and-a-web-camera/ – william wolly Sep 08 '18 at 14:00
  • The two images you're comparing would be the two frames of the video -- you'll need to capture that pixel array in a variable on each iteration, so you still have it around to compare it to the current video state on the next step. Something like [this](https://jsfiddle.net/qpxjcv3a/) but without the cross-origin error that I'm too rusty in canvas methods to remember how to work around... – Daniel Beck Sep 08 '18 at 14:06
  • I checked your jsfiddle. If you want to use this method you need to draw the actual video at the end of the interval, after setting globalCompositeOperation to "source-over". So that you will have the previous frame painted on the canvas when the interval is run again. – Burhan B Sep 08 '18 at 20:50

1 Answers1

1

As Daniel said you need to check the pixels, the legnth would be the same for all iterations. You should look into image hashing algorithms. At every interval you can calculate a hash, and store that in a global variable to compare at the next interval. This would also give you the option to set a threshold, so minor changes would not trigger motion detection.

This page explains image hashing in more details: https://jenssegers.com/61/perceptual-image-hashes

You can start by implementing average hash. It is quite simple. You would reduce your canvas size to 8x8 pixels. 

ctx.drawImage(video, 0, 0, video.width, video.height, 0, 0, 8, 8);
var imageData = ctx.getImageData(0, 0, 8, 8);
var data = imageData.data;

Then you iterate over the image data and calculate the brightness.

var brightnessdata = []
for(var i = 0; i<data.length; i+=4){
    brightnessdata.push((data[i]+data[i+1]+data[i+2])/3);
}

The rest is simply calculating the average brightness and comparing each pixel brightness to the average brightness to calculate the hash.

Burhan B
  • 130
  • 8
  • do you hash the canvas image? can you please explain a bit more what you mean by image hashing? – william wolly Sep 08 '18 at 17:49
  • It is called perceptual hashing. It works such that the changes in an image is reflected proportionally to the hash. Two similar images would produce similar hashes. The link I shared explains it rather well, I would have to copy paste the entire article to explain it here. But in the code I shared, you just need to get the average value of the brightnessdata array. Then compare each value in the array to the average, a higher value means 1 and a lower value means 0. – Burhan B Sep 08 '18 at 20:58