Getting canvas coordinates in python -For mac especially

Question

from keras.models import load_model
from tkinter import *
import tkinter as tk
import appscript
#import win32gui
from PIL import ImageGrab, Image
import numpy as np

model = load_model('mnist.h5')

def predict_digit(img):
    #resize image to 28x28 pixels
    img = img.resize((28,28))
    #convert rgb to grayscale
    img = img.convert('L')
    img = np.array(img)
    #reshaping to support our model input and normalizing
    img = img.reshape(1,28,28,1)
    img = img/255.0
    #predicting the class
    res = model.predict([img])[0]
    return np.argmax(res), max(res)

class App(tk.Tk):
    def __init__(self):
        tk.Tk.__init__(self)

        self.x = self.y = 0
        
        # Creating elements
        self.canvas = tk.Canvas(self, width=300, height=300, bg = "white", cursor="cross")
        self.label = tk.Label(self, text="Draw..", font=("Helvetica", 48))
        self.classify_btn = tk.Button(self, text = "Recognise", command = self.classify_handwriting)   
        self.button_clear = tk.Button(self, text = "Clear", command = self.clear_all)
       
        # Grid structure
        self.canvas.grid(row=0, column=0, pady=2, sticky=W, )
        self.label.grid(row=0, column=1,pady=2, padx=2)
        self.classify_btn.grid(row=1, column=1, pady=2, padx=2)
        self.button_clear.grid(row=1, column=0, pady=2)
        
        #self.canvas.bind("<Motion>", self.start_pos)
        self.canvas.bind("<B1-Motion>", self.draw_lines)

    def clear_all(self):
        self.canvas.delete("all")
        
    def classify_handwriting(self):
        HWND = self.canvas.winfo_id()  # get the handle of the canvas
        rect=self.canvas.coords(HWND)
        #rect = win32gui.GetWindowRect(HWND)  # get the coordinate of the canvas
        a,b,c,d = rect
        rect=(a+4,b+4,c-4,d-4)
        im = ImageGrab.grab(rect)

        digit, acc = predict_digit(im)
        self.label.configure(text= str(digit)+', '+ str(int(acc*100))+'%')

    def draw_lines(self, event):
        self.x = event.x
        self.y = event.y
        r=8
        HWND=self.canvas.create_oval(self.x-r, self.y-r, self.x + r, self.y + r, fill='black')
       
app = App()
mainloop()

Can anyone help me especially with this part :

HWND = self.canvas.winfo_id()  # get the handle of the canvas
        rect=self.canvas.coords(HWND)
        #rect = win32gui.GetWindowRect(HWND)  # get the coordinate of the canvas
        a,b,c,d = rect
        rect=(a+4,b+4,c-4,d-4)
        im = ImageGrab.grab(rect)

This is the error that appears :

Exception in Tkinter callback
Traceback (most recent call last):
  File "/Users/xxx/opt/anaconda3/lib/python3.8/tkinter/__init__.py", line 1883, in __call__
    return self.func(*args)
  File "<ipython-input-4-28fb2cc31a85>", line 52, in classify_handwriting
    a,b,c,d = rect
ValueError: not enough values to unpack (expected 4, got 0)

I am trying to create an ocr program to identify handwriting by letting the user draw a number and return the current value with high accuracy . But the problem faced was with the canvas and how to get the coordinates without using the win32gui module (Since im a mac user) so I'm looking for a solution or an alternative for win32gui method GetWindowRect

Try using `self.canvas.winfo_rootx()`, `self.canvas.winfo_rooty()`, `self.canvas.winfo_width()` and `self.canvas.winfo_height()` to calculate the bounding box. — acw1668, Jul 14 '21 at 12:37
Most of the time there is no need for `ImageGrab.grab`. Why are you using it? There might be a better option. — TheLizzard, Jul 14 '21 at 12:38
@acw1668 is this syntax correct ? a,b,c,d=HWND_rootx(), HWND_rooty(), HWND_width(),HWND_height() — AbAc, Jul 14 '21 at 13:51
@AbAc It looks like you are trying to take a screenshot of the canvas. But why are you doing that? Look at [this](https://stackoverflow.com/questions/9886274/how-can-i-convert-canvas-content-to-an-image) — TheLizzard, Jul 14 '21 at 13:57
@TheLizzard i want a capture so that i can compare it with other images in the database of numbers through the trained keras model — AbAc, Jul 14 '21 at 14:08
There are 2 things that you can do (other than what you are currently trying): tell `tkinter` to save the canvas as an image, then open the image or create an empty `PIL` image (with the correct size) and mirror all of the `self.canvas.create_oval` calls to it. I think that the second option would be easier to implement. — TheLizzard, Jul 14 '21 at 14:12

score 1 · Accepted Answer · answered Jul 14 '21 at 14:34

self.canvas.coords(HWND) is going to return the coordinates of an object on the canvas with the id or tag of whatever is in HWND. It does not return the coordinates of the canvas itself.

Since HWND doesn't represent an item on the canvas, self.canvas.coords(HWND) returns an empty list. That is why you get the error not enough values to unpack - your code requires four values but the list has zero.

If you are wanting the coordinates of the canvas on the physical screen there are other methods you can use. See the following code:

x, y = (self.canvas.winfo_rootx(), self.canvas.winfo_rooty())
width, height = (self.canvas.winfo_width(), self.canvas.winfo_height())

a, b, c, d = (x, y, x+width, y+height)

Yes this approach helps returning the canvas coordinates . THANKS — AbAc, Jul 14 '21 at 14:41

score 0 · Answer 2 · answered Sep 30 '22 at 12:43

Correct! It works fine on MACOS. comment the "import win32gui" line. Modify "classify_handwriting" function as below:

    def classify_handwriting(self):
        x, y = (self.canvas.winfo_rootx(), self.canvas.winfo_rooty())
        width, height = (self.canvas.winfo_width(), 
                         self.canvas.winfo_height())
        a, b, c, d = (x, y, x+width, y+height)
        im = ImageGrab.grab(bbox=(a,b,c,d))

Hope it works in your local setting!

Getting canvas coordinates in python -For mac especially

2 Answers2