Creating django objects with a random primary key

Question

I'm working with an API that wants me to generate opaque "reference IDs" for transactions with their API, in other words, unique references that users can't guess or infer in any way. (is 'infer' proper english?)

This is what I've hacked together currently:

randomRef = randint(0, 99999999999999)
while Transaction.objects.filter(transactionRef = randomRef).count():
    randomRef = randint(0, 99999999999999)

Transaction.objects.create(user=user, transactionRef=randomRef, price=999)

unfortunately my database seems to be missing transactions at the moment. I've realized that my method isn't particularly thread safe (say I'm running the same django code on multiple mod_wsgi apache threads, they could all be generating the same randomRef!)

Has anyone got a nicer trick to generate random primary keys for me?

score 16 · Accepted Answer · answered Jul 21 '10 at 00:23

16

Why not just encrypt the normal sequential ids instead? To someone who doesn't know the encryption key, the ids will seem just as random. You can write a wrapper that automatically decrypts the ID on the way to the DB, and encrypts it on the way from the DB.

answered Jul 21 '10 at 00:23

Amber

507,862
82
626
550

this is a great idea, thanks. I considered hashing a combination of userid and standard transaction.id, but that's not guaranteed to be unique either. can you suggest a good python function to encrypt with? I have installed m2crypto as a prerequisite for another package, but not really used it yet. – rdrey Jul 21 '10 at 02:30
Sorry @AlexBliskovsky, I'm a crypto n00b. Is there a way to specify the encrypted output to a small integer? Is it conceptually possible (to ensure there's no collision unless we make sure the domain of the plain text is smaller than the domain of the encrypted text)? – xster Apr 23 '12 at 20:29
http://stackoverflow.com/questions/4028998/symmetric-integer-to-integer-encryption – Alex Bliskovsky May 09 '12 at 11:38

score 15 · Answer 2 · answered Dec 17 '10 at 02:08

I created a gist based on this question: https://gist.github.com/735861

Following Amber's advice, the private keys are encrypted and decrypted using DES. The encrypted key is represented in base 36, but any other character-based representation will work as long as the representation is unique.

Any model that would need this kind of encrypted private key representation only needs to inherit from the model and manager shown in the code.

Here's the meat of the code:

import struct
from Crypto.Cipher import DES
from django.db import models

class EncryptedPKModelManager(models.Manager):
    """Allows models to be identified based on their encrypted_pk value."""
    def get(self, *args, **kwargs):
        encrypted_pk = kwargs.pop('encrypted_pk', None)
        if encrypted_pk:
            kwargs['pk'] = struct.unpack('<Q', self.model.encryption.decrypt(
                struct.pack('<Q', encrypted_pk)
            ))[0]
        return super(EncryptedPKModelManager, self).get(*args, **kwargs)


class EncryptedPKModel(models.Model):
    """Adds encrypted_pk property to children."""
    encryption = DES.new('8charkey') # Change this 8 character secret key

    def _encrypted_pk(self):
        return struct.unpack('<Q', self.encryption_obj.encrypt(
            str(struct.pack('<Q', self.pk))
        ))[0]

    encrypted_pk = property(_encrypted_pk)

    class Meta:
        abstract = True

For a Transaction object called transaction, transaction.encrypted_pk would return an encrypted representation of the private key. Transaction.objects.get(encrypted_pk=some_value) would search for objects based on the encrypted private key representation.

It should be noted that this code assumes only works for private keys that can be represented properly as long values.

score 6 · Answer 3 · answered Apr 23 '12 at 21:25

6

jbrendel created a class that you can simply inherit to get a custom ID.

https://github.com/jbrendel/django-randomprimary

answered Apr 23 '12 at 21:25

xster

6,269
9
55
60

1

It doesn't seem to work with django admin. It won't let me save a new record with a blank id because it's a required field; if I define an id then the random one doesn't get generated. Perhaps this only worked in a previous Django version? – sherbang Feb 28 '13 at 22:26
@sherbang To make it work with admin, just modify `if self.id:` part. If you look closely in the comments, it says " # Apparently, we know our ID already, so we don't have to # do anything special here." – shreks7 May 12 '14 at 07:07

score 3 · Answer 4 · answered Jul 21 '10 at 01:11

os.urandom(n) can "Return a string of n random bytes suitable for cryptographic use". Just make sure that n is large enough for 2**(8*n) to be well above the square of the number of "unique" keys you want to identify, and you can make the risk of collision as low as you want. For example, if you think you may end up with maybe a billion transactions (about 2**30), n=8 might suffice (but play it safe and use a somewhat larger n anyway;-).

Michael Williamson · Answer 5 · 2010-07-21T00:29:30.350

2

You should be able to set the transactionRef column in your database to be unique. That way, the database will not allow transactions to be added with the same transactionRef value. One possibility is to randomly generate UUIDs -- the probability of random UUIDs colliding is extremely small.

edited Jul 21 '10 at 00:29

answered Jul 21 '10 at 00:23

Michael Williamson

11,308
4
37
33

score 2 · Answer 6 · answered Jul 21 '10 at 00:25

2

Random integers are not unique, and primary keys must be unique. Make the key field a char(32) and try this instead:

from uuid import uuid4 as uuid
randomRef = uuid().hex

UUIDs are very good at providing uniqueness.

answered Jul 21 '10 at 00:25

Jesse Dhillon

7,841
1
34
34

1

Two UUIDs generated by uuid4 are not guaranteed to be unique, there's just an extremely small chance (but non-zero) chance of a collision. – Michael Williamson Jul 21 '10 at 00:27
thanks, I have never heard of UUIDs before, they seem much nicer than my current randint. should i still run things through a loop to make sure that the UUID is unused? – rdrey Jul 21 '10 at 02:34
Your database layer will throw an exception if you try to insert a non-unique key. You can catch that exception and make another UUID. Although when I posted it I thought this was a good answer, both Alex Martelli and Amber have given better suggestions if you can implement them. – Jesse Dhillon Jul 21 '10 at 02:52
Using a long char field for a primary key doesn't sound like a great idea. – Timmmm Jul 27 '12 at 14:35

Creating django objects with a random primary key

6 Answers6

Linked