detect mouse hover on overlapping + transparent images

Question

I'm building a little game where the user has buy items to furnish his house.

• I have a lot of items/images; so I decided to use, for each of them, a "matte" (an image) that would define the hoverable zone, rather than drawing a map area for each image.
  Example : here's the displayed couch, and its matte.
  I "convert" the matte into a canvas element, and will check later if the hovered pixel is transparent to detect if the item is hovered.

• The second thing is that a lot of items are overlapping, so I also need to check which layer is on top.

I have mousemove event (jQuery) on the house element; bound with the function getObjectsUnderMouse().

Basically, this is how getObjectsUnderMouse() works :

1. Get the mouse coordinates
2. Get the active (displayed) items in the house
3. Filter those items to keep only the ones where the mouse hits the canvas boundaries, knowing the item position and width/height)
4. Filter those items to keep only the ones where the mouse is NOT on a transparent pixel (canvas)
5. Filter those items to keep only the one on the top (z-index)
6. Give that item a mouseon class

I was quite happy with my code, which was quite a challenge but works perfectly on Chrome.

The problem I have is that it is slower elsewhere (not a so big deal), but; above all, seems to crash on ipad; and I need my game to run on ipad... :/

Does anyone knows why or have a better solution for this ?

Here's a demo of the game, and here's the javascript file where you can have a look at getObjectsUnderMouse().

Any advice is welcome !

Answer 1

Although a matte canvas contains the information you need to hit-test, keeping a full sized canvas for each matte is expensive in terms of memory. Keeping a canvas for each matte is likely using more resources than your iPad can handle.

Here's a way to greatly reduce your memory usage:

First, crop any extra transparent space out of each of your objects. For example, your couch is 600x400=240000 pixels, but cropping away the empty space shrinks the image to 612x163=99756 pixels. That's a savings of 58% over the original image size. Less pixels means less memory for a matte.

Instead of keeping a full-sized canvas for each object, instead keep an array for each object which only contains the opacity of each pixel in that image. An array value of 1 indicates that pixel is opaque (and is part of the object). An array value of 0 indicates that pixel is transparent (no part of the object is at this pixel).

Then hit-test against the pixel array instead of hit-testing against a matte canvas.

If you test the arrays in z-index order, you can even tell which object is on top of another object.

Here's example code and a Demo:

var canvas=document.getElementById("canvas");
var ctx=canvas.getContext("2d");
var cw=canvas.width;
var ch=canvas.height;
var $canvas=$("#canvas");
var canvasOffset=$canvas.offset();
var offsetX=canvasOffset.left;
var offsetY=canvasOffset.top;

// display which object the mouse is over
var $result=$('#result');

// create an array of target objects
var targets=[];
targets.push({ name:'couch', x:25, y:50, hitArray:[], url:'https://dl.dropboxusercontent.com/u/139992952/multple/couch.png' });
targets.push({ name:'lamp', x:50, y:30, hitArray:[], url:'https://dl.dropboxusercontent.com/u/139992952/multple/lamp.png' });
var imgCount=targets.length;

// load the image associated with each target object
for(var i=0;i<targets.length;i++){
  var t=targets[i];
  t.image=new Image();
  t.image.crossOrigin='anonymous';
  t.image.index=i;
  t.image.onload=start;
  t.image.src=t.url;   
}

// this is called when each image is fully loaded
function start(){

  // return if all target images are not loaded
  if(--imgCount>0){return;}

  // make hit arrays for all targets
  for(var i=0;i<targets.length;i++){
    var t=targets[i];
    t.hitArray=makeHitArray(t.image);
  }

  // resize the canvas back to its original size
  canvas.width=cw;
  canvas.height=ch;   

  // draw all targets on the canvas
  for(var i=0;i<targets.length;i++){
    var t=targets[i];
    t.width=t.image.width;
    t.height=t.image.height;
    ctx.drawImage(t.image,t.x,t.y);
  }

  // listen for events
  $("#canvas").mousemove(function(e){handleMouseMove(e);});

}

// Draw a target image on a canvas
// Get the imageData of that canvas
// Make an array containing the opacity of each pixel on the canvas
// ( 0==pixel is not part of the object, 1==pixel is part of the object)
function makeHitArray(img){
  var a=[];
  canvas.width=img.width;
  canvas.height=img.height;
  ctx.drawImage(img,0,0);
  var data=ctx.getImageData(0,0,canvas.width,canvas.height).data;
  for(var i=0;i<data.length;i+=4){
    // if this pixel is mostly opaque push 1 else push 0
    a.push(data[i+3]>250?1:0);
  }
  return(a);
}


function handleMouseMove(e){

  // tell the browser we're handling this event
  e.preventDefault();
  e.stopPropagation();

  // get the mouse position
  mouseX=parseInt(e.clientX-offsetX);
  mouseY=parseInt(e.clientY-offsetY);

  // Test the mouse position against each object's pixel array
  // Report hitting the topmost object if 2+ objects overlap
  var hit='Not hovering';
  for(var i=0;i<targets.length;i++){
    var t=targets[i];
    var imgX=mouseX-t.x;
    var imgY=mouseY-t.y;
    if(imgX<=t.width && imgY<=t.height){
      var hitArrayIndex=imgY*t.width+imgX;
      if(hitArrayIndex<t.hitArray.length-1){
        if(t.hitArray[hitArrayIndex]>0){
          hit='Hovering over '+t.name;
        }
      }     
    }
  }

  $result.text(hit);

}

body{ background-color: ivory; padding:10px; }
#canvas{border:1px solid red;}

<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<h4 id='result'>Move mouse over objects.</h4>
<canvas id="canvas" width=450 height=250></canvas>