The setup
As part of a web vector editing tool written in Javascript, I'm implementing hit testing using a hit canvas strategy similar to that of Concrete.js. For most of it, it works pretty well: I'm drawing my shapes twice (once on the display canvas, and once on a hit canvas).
When querying the canvas, I check the hovered pixel of the hit canvas and extract interaction information (i.e. which object id is stored there).
The problem
This works well inside shapes, but is painfully flawed at the boundaries where anti-aliasing makes the stored data invalid (it gets mixed with whatever background data was there before). Are there good strategies for dealing with this data boundary problem?
Without disabling anti-aliasing for canvas methods, then we're bound to have some boundary regions across overlapping regions that will store merged data from multiple regions.
The simple scenario
In a binary scenario (some foreground vs the background), then this can be mitigated as we can assume the background to have no value, and any value becomes some foreground.
The real scenario
However, in a general scenario with multiple shapes overlapping each other on top of the background, is there any reasonable strategy for error detection? (or error correction, but I assume that's harder)
If I can tell that the data is invalid (i.e. it consists of perturbed data due to anti-aliasing), then I can use a different strategy for those few pixels at the boundaries. But I feel that it's impossible to tell whether the data I'm extracting is valid in the general scenario where we can have many overlapping shapes.
Of course, one solution is to NOT use a hit canvas. But I was wondering whether people had found a solution using hit canvases since they seem great for dealing with complex geometries.
Anti-aliasing
The ideal solution would be to disable anti-aliasing, which I don't think is possible for canvas methods [*].
[*] I know we can disable filtering when rendering images (e.g. that question) such as with imageSmoothingEnabled=false or rescaling with image-rendering: pixelated, but those don't solve the problem of anti-aliasing when drawing shapes / paths.
