Overflowing ints in python

Question

I'm converting a piece of Ian Bell's Elite from C to Python. The C (ab)uses 16-bit integers, which overflow. Python uses magical Python integers, which do not. This is a problem.

I could just define a class UInt16: (UPDATE: Or use ctype's built in UInt16) which behaves like I want it to, but then I'd have to "force" every number in the entire program to be a UInt16. Is there any way I can tell my interpreter "Hey, you know Int? Well, I'd like you to use this class every place you would normally use one of those."

Or, failing that, is there another easy way I could implement integer overflow?

The correct answer really here is "don't abuse integers". Just manually implement the "if greater than 2^16-1, subtract 2^16-1" manually. Of course, this will mean quite a few code changes, but I'd argue it's worth it for the sake of readability and defined behaviour. — slugonamission, May 25 '14 at 18:35
just a tip: type(huge_number) will return 'long' (the magic integer) if the number is greater than 2^16-1. — Roy Iacob, May 25 '14 at 18:36
@slugonamission You might wanna use modulus instead, like `n%(2**16)`. But Jonathon is correct, ctypes' uint16 should work. — cjfaure, May 25 '14 at 18:37
@slugonamission Well, call up Ian and tell him that. I'm stuck with it. — Schilcote, May 25 '14 at 18:39
@Trimsty - whoops. My brain isn't fully in gear today. Cheers :) — slugonamission, May 25 '14 at 18:40
@RoyIacob: There's no longer a distinction between regular and long integers in Python 3. They're all `int`s now. — Blckknght, May 25 '14 at 20:08
I don't see a real benefit to using `ctypes.c_uint16`. The only number method implemented is `nb_bool` (i.e. `__bool__`). I'd use a `UInt16` class similar to this [`Int4` class](http://stackoverflow.com/a/23709473/205580). — Eryk Sun, May 25 '14 at 22:54
For posterity's sake; modulo by `2**n` where n is your number of bits simulates overflow perfectly. — Schilcote, Jul 24 '15 at 20:32

score 3 · Accepted Answer · edited May 23 '17 at 12:06

Short answer: it is not possible to override default integer behaviour as you would desire.

Longer answer:

Firstly, we can see that several implementations for 16-bit unsigned integer exists in Python:

In built-in library ctypes 16-bit unsigned integer is available as c_uint16
Numpy library also contains unint16
Implementing your own uint16 data type (with operator overloading etc.)

The question whether we can use these or some other data types to override default integer in Python is trickier.

Let's assume that user creates all integers in the program using int method. If this is the case, then we can wrap the int method to create unsigned integer instead of regular integer:

from ctypes import c_uint16
_int = int

def int(x):
    return c_uint16(x)

For example the following code works just fine:

a = int(1)

As we can inspect:

>>> type(a)
<class 'ctypes.c_ushort'>

By here comes the problem:

b = 1

In this case int method is not invoked, but instead, b is simply assigned value 1. What we would wish to do would be to override this behaviour too, but unfortunately it is not possible.

Not an answer. I'm looking for a way to make Python use the `uint16`s in place of normal ints without explicitly stating such every time I use a number. — Schilcote, May 25 '14 at 18:42

score 1 · Answer 2 · edited May 23 '17 at 12:14

AFAIR its impossible. You would like to monkey-patch built-in type, which is impossible in python.

Extended version

Integer literals (1, not int("1")) are always of type int from builtins/__builtins__ module (depending on python version). What you would like to do is basically copy all methods on unint16 to int class (as method are just attributes of function type). In this case in every place you've stated 1, you'd get an object of type int, which behaves just like unint16.

This is impossible in python. I suppose that underlying implementation is getting easier because of this, but there is deeper logic behind this.

int is one of core classes, like list, str, object, set, dict, tuple, etc. Suppose that someone created VERY handy library that almost everyone would use. If you've ever used Java, then imagine impact on a scale of Apache Commons or Spring. Virtually everyone that counts would use that, because that stuff makes development faster. Now, suppose that that someone created fork - used internally in his company, saying that he done that to keep codebase stable - that has a little piece of code, that changes builtin object behaviour, so that it sends some crucial data to him. He could scam millions of users on their private data (addresses, which he will then sell, passwords, whatever). Creepy scenario, right?

Now, this doesn't seem very real, but what if you let end-user evaluate python scripts? You probably have seen text editors like gedit) or music players (Rhythmbox) with builtin python shell, providing interface to program internals. Now, soemone, somehow inject change of object behaviour to your machine. Another creepy scenario.

Those are examples that came to my mind, I can't really remember what did python's creators say about it - that's why I provided link to further discussion.

A little off-topic - this actually MAY lead to some fine libraries ("this" meaning "allowing monkey-patching on builtins"). Imagine monkey patching all bultins so that they are stored in some multiprocessing manager. You virtually get java-style multithreading based on multiprocessing and can somehow avoid GIL. Of course, there are many other reasons why this would not be REALLY good ("illusion of multithreading is not multithreading", for example), but I'd find it useful for simple everyday desktop scripting.

Overflowing ints in python

2 Answers2