How to prevent user from typing 'e' in input number in firefox

Question

Suppose that I have an input number like this :

<input id="my-input" type="number">

I coded something that prevented the user from typing the letter 'e' (and dots, commas) in that input

$('my-input').on('keypress', (event) => {

    if(!isIntegerChar()) {
        event.preventDefault();
    }

    function isIntegerChar() {
        return /[0-9]/.test(
            String.fromCharCode(event.which))
    }
});

This code worked perfectly until recently when I noticed that I cannot do backspace, arrow up, arrow down, arrow left and arrow right in firefox (Yes just firefox. In the other browsers, this code works like a charm).

Is there a way to make it work also in firefox ? (prevent the user from typing the letter 'e' and while leaving backspace, arrow up, arrow down, arrow left, arrow right).