You can use the onkeypress
event to limit just the characters you want to accept.
This doesn't validate the special characters, it doesn't allow the user to type them. Here is a site that will tell you the keycode: http://keycode.info/
For example, this will just allow numbers:
var keyCode = event.keyCode;
if((keyCode >= 48 && keyCode <= 57) ||keyCode == 8){
event.returnValue = true;
}else{
event.returnValue = false;
}
You could also reverse this and selectively exclude special characters by their keycode.
Here is a picture of how I do it. I don't use JQuery like the link in the comments, although that certainly works too:

One caveat: For use in a Edit box (in XPages), the keypad characters work the same even though they have different keycodes. In some cases, like a Dojo Comboxbox, you need to also specify both the normal numbers, and the keypad numbers. keyCode values for numeric keypad?