I have a bit of a complicated question (I did a lot of research but wasn't able to find what I'm looking for), basically I'm building a labeling tool where i get a set of images and i want to be able to click the corners of objects and create a point where the user clicks.
Few things to note (I've already done these)
- Images can be any orientation and i need to rotate them (rotate from an orientation)
- The image should start out scaled to fit the canvas (setting a scale to "zoom out" from image and canvas sizes)
- Users can "pan" around (translation based on arrow keys)
- Users can zoom in and out on the image (scale with shift + arrow up/down)
- Users can reset an image back to center (spacebar centers, shift + spacebar resets zoom initial and re-centers)
The issue I have now is I'm building the click portion (where I draw a point at the cursor location). I have tried multiple things to put the mouse coordinates at the correct location (accounting for scale, translation and rotation) and I'm having a hard time wrapping my head around it. Would love some help or pointers as to how to basically inverse the rotation, scale and translation I've applied to get the point in the correct place.
To give some real context around this I made a Codepen to show whats happening.
Codepen to see it live with the arrow keys / clicks on the canvas
const red = '#ff0000';
class App extends React.Component<{}, {}> {
private canvas: HTMLCanvasElement
private image = new Image
private ctx: CanvasRenderingContext2D | null
private data: any
private orientation: number = 270
private moveKeys: {[key: number]: number} = {}
private cw: number
private ch: number
private scaleFactor: number = 1.00
private startX: number
private startY: number
private panX: number
private panY: number
private isShiftPressed: boolean
private defaultScaleFactor: number = 1.00
private imagePoints: number[][] = []
loadImage = (url: string) => {
this.image.onload = () => {
const iw = this.orientation === 0 || this.orientation === 180 ? this.image.width : this.image.height
const ih = this.orientation === 0 || this.orientation === 180 ? this.image.height : this.image.width
const smaller = Math.min(this.canvas.width / iw, this.canvas.height / ih)
this.defaultScaleFactor = smaller
this.scaleFactor = smaller
}
this.image.src = 'https://i.stack.imgur.com/EYvnM.jpg'
}
componentWillUnmount() {
document.removeEventListener('keyup', this.handleKeyUp)
document.removeEventListener('keydown', this.handleKeyDown)
// window.removeEventListener('resize', this.resizeCanvas)
this.canvas.removeEventListener('click', this.handleCanvasClick)
}
componentDidMount() {
this.isShiftPressed = false
document.addEventListener('keyup', this.handleKeyUp)
document.addEventListener('keydown', this.handleKeyDown)
// window.addEventListener('resize', this.resizeCanvas) // dont need for this example
requestAnimationFrame(this.animate)
const elem = document.getElementById('canvasContainer')
if (!elem) return
const rect = elem.getBoundingClientRect()
this.canvas.addEventListener('click', this.handleCanvasClick)
this.canvas.width = rect.width
this.canvas.height = rect.height
this.ctx = this.canvas.getContext('2d')
this.cw = this.canvas.width
this.ch = this.canvas.height
this.startX = -(this.cw / 2)
this.startY = -(this.ch / 2)
this.panX = this.startX
this.panY = this.startY
this.loadImage()
}
handleCanvasClick = (e) => {
let rect = this.canvas.getBoundingClientRect()
let x = e.clientX - rect.left
let y = e.clientY - rect.top
this.imagePoints.push([x, y])
}
animate = () => {
Object.keys(this.moveKeys).forEach( key => {
this.handleMovement(key, this.moveKeys[key])
})
this.drawTranslated()
requestAnimationFrame(this.animate)
}
handleMovement = (key, quantity) => {
const moveUnit = 20
switch (parseInt(key)) {
case 32: // spacebar
this.panX = this.startX
this.panY = this.startY
if (this.isShiftPressed) {
this.scaleFactor = this.defaultScaleFactor
}
break
case 37: // left
if (this.orientation === 90 || this.orientation === 270) {
this.panY -= moveUnit
} else {
this.panX -= moveUnit
}
break
case 38: // up
if (this.isShiftPressed) {
this.scaleFactor *= 1.1
} else {
if (this.orientation === 90 || this.orientation === 270) {
this.panX += moveUnit
} else {
this.panY += moveUnit
}
}
break
case 39: // right
if (this.orientation === 90 || this.orientation === 270) {
this.panY += moveUnit
} else {
this.panX += moveUnit
}
break
case 40: // down
if (this.isShiftPressed) {
this.scaleFactor /= 1.1
} else {
if (this.orientation === 90 || this.orientation === 270) {
this.panX -= moveUnit
} else {
this.panY -= moveUnit
}
}
break
default:
break
}
}
handleKeyUp = (e) => {
if (e.shiftKey || e.keyCode === 16) {
this.isShiftPressed = false
}
delete this.moveKeys[e.keyCode]
}
handleKeyDown = (e) => {
e.preventDefault()
if (e.shiftKey || e.keyCode === 16) {
this.isShiftPressed = true
}
e.keyCode in this.moveKeys ? this.moveKeys[e.keyCode] += 1 : this.moveKeys[e.keyCode] = 1
}
drawTranslated = () => {
if (!this.ctx) return
const ctx = this.ctx
ctx.clearRect(0, 0, this.cw, this.ch)
ctx.save()
ctx.translate(this.cw / 2, this.ch / 2)
ctx.rotate(this.orientation * Math.PI / 180)
ctx.scale(this.scaleFactor, this.scaleFactor)
ctx.translate(this.panX, this.panY)
const transformedWidth = this.canvas.width / 2 - this.image.width / 2
const transformedHeight = this.canvas.height / 2 - this.image.height / 2
ctx.drawImage(
this.image,
transformedWidth,
transformedHeight
)
const pointSize = 10
if (this.imagePoints.length > 0) {
this.imagePoints.forEach( ([x, y]) => {
ctx.fillStyle = red
ctx.beginPath()
// Obviously the x and y here need to be transformed to work with the current scale, rotation and translation. But I'm stuck here!
ctx.arc(x, y, pointSize, 0, Math.PI * 2, true)
ctx.closePath()
ctx.fill()
})
}
ctx.restore()
}
handleResetUserClicks = () => {
this.imagePoints = []
}
render() {
return (
<div id="container">
<div>Use arrow keys to pan the canvas, shift + up / down to zoom, spacebar to reset</div>
<div id="canvasContainer">
<canvas ref={this.assignCameraRef} id="canvasElement" style={{ position: 'absolute' }} ref={this.assignCameraRef} />
</div>
<div>
<button onClick={this.handleResetUserClicks}>Reset Clicks</button>
</div>
</div>
)
}
assignCameraRef = (canvas: HTMLCanvasElement) => this.canvas = canvas
}
Please ignore the lack of defined props and the few hardcoded values (like orientation). I removed a bit of code and abstracted this to be more generic and part of that meant hardcoding the image url to a dummy one I found online and setting some of the parameters for that image as well.
