Securely encrypt integers (up to 2^48) into the shortest possible URL-safe string

Question

In my Django application I have hierarchical URL structure:

webpage.com/property/PK/sub-property/PK/ etc...

I do not want to expose primary keys and create a vulnerability. Therefore I am encrypting all PKs into strings in all templates and URLs. This is done by the wonderful library django-encrypted-id written by this SO user. However, the library supports up to 2^64 long integers and produces 24 characters output (22 + 2 padding). This results in huge URLs in my nested structure.

Therefore, I would like to patch the encrypting and decrypting functions and try to shorten the output. Here is the original code (+ padding handling which I added):

# Remove the padding after encode and add it on decode
PADDING = '=='

def encode(the_id):
    assert 0 <= the_id < 2 ** 64

    crc = binascii.crc32(bytes(the_id)) & 0xffffff

    message = struct.pack(b"<IQxxxx", crc, the_id)
    assert len(message) == 16

    cypher = AES.new(
        settings.SECRET_KEY[:24], AES.MODE_CBC,
        settings.SECRET_KEY[-16:]
    )

    return base64.urlsafe_b64encode(cypher.encrypt(message)).rstrip(PADDING)


def decode(e):
    if isinstance(e, basestring):
        e = bytes(e.encode("ascii"))

    try:
        e += str(PADDING)
        e = base64.urlsafe_b64decode(e)
    except (TypeError, AttributeError):
        raise ValueError("Failed to decrypt, invalid input.")

    for skey in getattr(settings, "SECRET_KEYS", [settings.SECRET_KEY]):
        cypher = AES.new(skey[:24], AES.MODE_CBC, skey[-16:])
        msg = cypher.decrypt(e)

        crc, the_id = struct.unpack("<IQxxxx", msg)

        if crc != binascii.crc32(bytes(the_id)) & 0xffffff:
            continue

        return the_id
    raise ValueError("Failed to decrypt, CRC never matched.")

# Lets test with big numbers
for x in range(100000000, 100000003):
    ekey = encode(x)
    pk =  decode(ekey)
    print "Pk: %s Ekey: %s" % (pk, ekey)

Output (I changed the strings a bit, so don't try to hack me :P):

Pk: 100000000 Ekey: GNtOHji8rA42qfq3p5gNMI
Pk: 100000001 Ekey: tK6RcAZ2MrWmR3nB5qkQDe
Pk: 100000002 Ekey: a7VXIf8pEB6R7XvqwGQo6W

I have tried to modify everything in the encode() function but without any success. The produced string has always the length of 22.

Here is what I want:

• Keep the encryption strength near to the original level or at least do not decrease it dramatically
• Support integers up to 2^48 (~281 trillions), or 2^40, because as it is now with 2^64 is too much, I do not think that we will ever have such huge PKs in the database.
• I will be happy with string length between 14-20. If its 20.. then yeah, its still 2 chars less..

Answer 1

Currently you are using CBC mode with a static IV, so the code you have isn't secure anyway and, like you say, produces rather large ciphertexts.

I would recommend swapping from CBC mode to CTR mode, which lets you have a variable length IV. The normal recommended length for the IV (or nonce) in CTR mode, I think, is 12, but you can reduce this up or down as needed. CTR is also a stream cipher which means what you put in is what you get out in terms of size. With AES, CBC mode will always return you ciphertexts in blocks of 16 bytes so even if you are encrypting 6 bytes, you get 16 bytes out, so isn't ideal for you.

If you make your IV say... 48 bits long and aim to encrypt no larger than 48 bits, you'll be able to produce a raw output of 6 + 6 = 12 bytes, or with base64, (4*(12/3)) = 16 bytes. You will be able to get a lower output than this by further reducing your IV and/or input size (2^40?). You can lower possible values of your input as much as you want without damaging the security.

Keep in mind that CTR does have pitfalls. Producing two ciphertexts that share the same IV and key means that they can be trivially broken, so always randomly generate your IV (and don't reduce it in size too much).