1

I want to subclass int (or some other analogous builtin numerical type), that I can explicitly type check.

This q&a is similar, but didn't answer what I'm seeing exactly: Sub-classing a built-in Python type such as int

Here's a rough example of what I'm trying to achieve:

class MyId( int ) : pass 

class MyCollection() :

    def __init__( self ):
        self.__id = MyId( 0 )

    def nextId( self ) :        
        self.__id+=1
        print "isinstance",  isinstance(self.__id, MyId)
        return self.__id

Unfortunately, my invocation of isinstance returns False. How do I make it succeed (ideally with this same basic concept)? It's obvious how to achieve is this by giving MyId class a "has a" rather than "is a" relationship with int... but I thought it be nicer to just make it an int with a specific "name".

Currently, I'm writing this in Py2, but any cross version answers are appreciated if applicable.

martineau
  • 119,623
  • 25
  • 170
  • 301
BuvinJ
  • 10,221
  • 5
  • 83
  • 96
  • 3
    If you don't want to implement all the relevant magic methods (`__add__, __radd__, etc.`) in `MyId`, you could just go with `self.__id = MyId(self.__id + 1)`. – user2390182 Nov 30 '18 at 16:47
  • 2
    If your goal is to type check, why not just just check that `__id` is an `int`? `isinstance(self.__id, int)` – axblount Nov 30 '18 at 16:49
  • 3
    I agree with @axblount as I don't really understand here the need of overriding the builtin int. – Corentin Limier Nov 30 '18 at 17:02
  • Thanks, @schwobaseggl . That's definitely in the running for the "best" solution. It's shorter, but I hate that it requires a client class having to understand what equates to a private implementation detail. I also don't care for destroying an object and creating a new one, just to change a value. Essentially, we have an immutable int in that case. – BuvinJ Nov 30 '18 at 18:58
  • As I wrote in another comment here, btw: The point is to determine whether some polymorphic functions are returning and/or dealing with a specific kind of object, because the context of what to do with such will hinge on that. I need to differentiate between an int and this "id type". – BuvinJ Nov 30 '18 at 19:01
  • @BuvinJ for this case I would use composition instead of inheritance. An id is not a special integer. Dividing an id makes no sense for example. – Corentin Limier Nov 30 '18 at 20:39
  • Yeah. I ended up doing that. I just thought I could take a really short and sweet approach. Rather than wrapping an `int`, I figured I could just a derived class. – BuvinJ Nov 30 '18 at 20:45

4 Answers4

3

That's because you need to override the __add__ method.

If you don't override this method, it will use the builtin int __add__ method which returns a new integer object.

See this topic which explains this behavior as mentioned by @martineau in comments.

class MyId( int ):
    def __add__(self, i):
        return MyId(super(MyId, self).__add__(i))

class MyCollection() :

    def __init__( self ):
        self.__id = MyId( 0 )

    def nextId( self ) :        
        self.__id += 1
        print "isinstance",  isinstance(self.__id, MyId)
        return self.__id

a = MyCollection()
a.nextId()

Prints: isinstance True

martineau
  • 119,623
  • 25
  • 170
  • 301
Corentin Limier
  • 4,946
  • 1
  • 13
  • 24
  • Would be a better answer if you explained _why_ `__add__()` needs to be implemented. – martineau Nov 30 '18 at 17:15
  • Hmm. This is definitely a valid answer, I upvoted it. But, I'm not in love with it. I hate the idea that I have to redefine add the functions that I'm trying to get for free. That's the point of the sub classing. @schwobaseggl answer (posted as a comment) might be better... But it's less than perfect too I think... – BuvinJ Nov 30 '18 at 18:56
  • Am I going to have to override equality operators too? ... If so I'm not sure that I'm gaining anything by using int as a base... I already saw couldn't print one of these objects implicitly... – BuvinJ Nov 30 '18 at 19:16
  • 1
    @martineau that's because it is not totally clear in my head. Is that because int is immutable, so `__add__` needs to build a new (int) object ? – Corentin Limier Nov 30 '18 at 20:24
  • 1
    @BuvinJ you don't have to override equality operators, just operations that returns an "updated" integer. – Corentin Limier Nov 30 '18 at 20:34
  • Thanks @CorentinLimier. Note that I could not print the object, nor format it with %d... There were a series of built-in `int` features that just wouldn't work. I find that terribly strange. See my "answer" for a somewhat more comprehensive definition of the sort of needs I have for the object. – BuvinJ Nov 30 '18 at 20:41
  • In the real use case, I have functions which can return multiple types. One of them, is this id, which indicates that the real answer will be provided asynchronously. There is a whole other set of mechanisms involving in multiple producers and consumers using this "collection" and "id"... – BuvinJ Nov 30 '18 at 20:42
  • I second the question on immutability. Is an `int` immutable? It makes since that a string is, since that involves reallocating more memory, etc. potentially, but can't an int just be overwritten in the same memory location? – BuvinJ Nov 30 '18 at 20:47
  • @CorentinLimier. So, it turns out ints are immutable in Python. I didn't realize that. Here an SO post explaining why: https://stackoverflow.com/questions/29431314/why-python-data-types-are-immutable – BuvinJ Nov 30 '18 at 21:30
  • Corentin: No, it's because without defining the method for the `MyId` class the one inherited from the built-in `int` class gets used—so the `self.__id += 1` statement effectively turns `self.__id` into an `int`. You can verify this puttng a `print "isinstance", isinstance(self.__id, MyId)` _before_ and after it. – martineau Nov 30 '18 at 22:10
  • @martineau yeah I knew that but I was wondering why the builtin `__add__`was returning a new int object but I realize now it has nothing to do with immutability : I don't have an example of `__add__` function that does not return a new object. – Corentin Limier Nov 30 '18 at 22:21
  • Corentin: It's complicated. See the answers (and the many excellent comments) to a question I asked [once](https://stackoverflow.com/questions/11836570/how-to-implement-iadd-for-an-immutable-type), that **almost** makes this question a duplicate of it (once you realize that it's the `+=` messing things up, that is). – martineau Nov 30 '18 at 23:06
  • Corentin: I now think your answer would be more correct if it implemented `__iadd__()` — but I upvoted it anyway because it's on the right track. – martineau Nov 30 '18 at 23:16
  • @martineau thanks for the link I added it to the answer. Will help OP and others to dig into the subject. – Corentin Limier Nov 30 '18 at 23:25
1

It sounds like what you're after is being able to check that values being passed around have been created in a specific way. In Python 3.5.2+ there is the typing module that provides NewType. This allows you to do static analysis of your code to make sure it's doing the things you expect it to do. The example given in the documentation is:

from typing import NewType

UserId = NewType('UserId', int)
some_id = UserId(524313)

The static type checker will treat the new type as if it were a subclass of the original type. This is useful in helping catch logical errors:

def get_user_name(user_id: UserId) -> str:
    ...

# typechecks
user_a = get_user_name(UserId(42351))

# does not typecheck; an int is not a UserId
user_b = get_user_name(-1)

No actual type checking is performed at runtime, and the value returned by NewType is just a pass-through function that returns its argument unchanged. This also means you cannot do things like isinstance(obj, UserId), since UserId is not an actual class. What is does mean is, as mentioned by the documentation, static type checkers will help uncover logical errors -- which seems like what you're after.

Dunes
  • 37,291
  • 7
  • 81
  • 97
  • I like this idea, but it doesn't create a separate type. `type(UserId(1)) == int` is true. `UserId` itself is a function not a type. – axblount Nov 30 '18 at 17:46
0

Instead of subclassing int just check that your instance variable is an int.

class MyCollection():

    def __init__( self ):
        self.__id = 0

    def nextId( self ) :        
        self.__id += 1
        print "isinstance",  isinstance(self.__id, int)
        return self.__id
axblount
  • 2,639
  • 23
  • 27
  • Thanks, but that doesn't serve my needs. The point is to determine whether some polymorphic functions are returning and/or dealing with a specific kind of object, because the context of what to do with such will hinge on that. I need to differentiate between an int and this "id type" – BuvinJ Nov 30 '18 at 18:40
  • Then I think a 'has a' relationship is your best. It sounds like you don't want `__id` to be treated like an int. The best way to prevent that is by creating a new class. – axblount Nov 30 '18 at 18:47
  • I guess I need to go with a damn "has a" after all. See my (displeasing...) answer. – BuvinJ Nov 30 '18 at 19:59
0

Per Dunes' suggestion, I simply dropped the entire int concept entirely. As he pointed out, any vanilla object can implicitly be used as a unique key!

In fact MyId could be defined as simply: class MyId: pass. Often, that would be it - a perfectly usable, implicitly unique key!

For my use case, however, I need to pass these keys back and forth across sub processes (via multiprocessing queues). I ran into trouble with that ultra light weight approach, as the hash value would change when the objects where pickled and pushed across processes. A minor secondary concern was that I wanted to make these objects easy to log and manually read / match up through logs. As such, I went with this:

class _MyIdPrivate: pass
class MyId :
    def __init__( self ):
        self.__priv = _MyIdPrivate() 
        self.__i = hash( self.__priv )            
    def __str__( self ): return str( self.__i )
    def __hash__( self ): return self.__i
    def __eq__( self, other ): 
        try: return self.__i == other.__i
        except: return False     

class MyCollection :

    def __init__( self ):
        self.__objs={}

    def uniqueId( self ): return MyId()

    def push( self, i, obj ):
        self.__objs[ i ] = obj     

    def pop( self, i ):
        return self.__objs.pop( i, None )     

c = MyCollection()
uId = c.uniqueId()
print "uId", uId
print "isinstance", isinstance(uId, MyId)
c.push( uId, "A" )
print c.pop( MyId() )
print c.pop( uId )

As you can see, I wrapped the short and sweet approach into a more comprehensive/verbose one. When I create the MyId object, I create a _MyIdPrivate member, and take the hash of that at that moment of creation. When pickling, and pushing across sub projects, that _MyIdPrivate hash will change - but it doesn't matter because I captured the initial value, and everything ends up pivoting off of that.

The main benefit of this approach over the original int plan is that I get a unique key without "calculating" or assigning it directly.

As Dunes' suggested I could have also used a uuid. I can see pros and cons to that vs this...

BuvinJ
  • 10,221
  • 5
  • 83
  • 96
  • Combining `__hash__` and your `increment` method is a very bad idea. Objects should not have mutable hash values, or at the very least should not be mutated whilst they are keys in a mapping. If do you need mutate the hash value of a key then you will not be able to reference it any more. See https://docs.python.org/3/reference/datamodel.html#object.__hash__ for more details. – Dunes Nov 30 '18 at 21:34
  • Good point, @Dunes! I can see the logical flaw in what I have here. Can you suggest a specific rewrite for what I'm trying to achieve? – BuvinJ Nov 30 '18 at 21:37
  • I revised and re-posted. Better? – BuvinJ Nov 30 '18 at 22:01
  • I don't quite get why you want what you want. It's a bit of an anti-pattern for python -- the "if it acts/quacks like a duck" thing. I get the impression you're not overly familiar with Python and are trying to force it to behave more like a statically typed language. Your new implementation looks like it will work and is along the lines of what I would have suggested to make this work. – Dunes Nov 30 '18 at 22:09
  • I am quite familiar with Python. The entire point to all this is the fact that it is a weakly typed language. I have a situation where it makes perfect sense for some functions to return and or handle multiple types, one of which is a key to collection. – BuvinJ Nov 30 '18 at 22:17
  • I need to differentiate between an `int` and key (which is an `int` essentially). This is why, all I wanted to do was sub class `int` in a single line. Now, I'm forced into way more ugliness than I wanted, because that has proved to not work in several critical matters. – BuvinJ Nov 30 '18 at 22:20
  • I can think of multiple alternate solutions to my use case potentially, but a quick type checking should be by far the easiest. – BuvinJ Nov 30 '18 at 22:23
  • You still haven't said why your key needs to be an int. What functionality of an int does it need? If you just need a unique identifier with a distinct class that can be hashed, then `class MyId(object): pass` should suffice. And then `nextId` can just return `MyId()`. Your keys will be guaranteed unique whilst they are still reachable. – Dunes Nov 30 '18 at 23:38
  • re: typing. Strong/Weak typing as terms aren't overly useful as most languages include aspects of both, including Python and C/C++. Python is a dynamically type checked, type safe and memory safe language. Whereas C/C++ are statically type checked, not type safe and not memory safe. – Dunes Nov 30 '18 at 23:43
  • Brilliant! You are totally right. Thanks so much, @Dunes! I've revised the answer. – BuvinJ Dec 01 '18 at 00:23
  • @Dunes, so now I have problem, perhaps you can answer. When I push this across processes using mutliprocessing, it seems that I lose the hash! I need these keys to cross back forth across such. I did have that with `int` approach. Is there a serialize/deserialize way to handle this? – BuvinJ Dec 01 '18 at 01:25
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/184557/discussion-between-dunes-and-buvinj). – Dunes Dec 01 '18 at 12:37