0

I need a way to transmit my item IDs to the browser in a way that prevents spidering/crawling the items.

My thought is to encrypt the integer IDs with a secret and shared salt. This allows a large number of permanent but unpredictable URLs for the same unique item. Suppose I need to transmit results for records 1 and 2. Rather than transmitting IDs in the clear:

{
    1: "Item One",
    2: "Item Two"
}

I'll first encrypt the IDs at the web server:

string RESULT_SET_SALT = "randomValue1";
foreach(ResultItem item in results) {
    item.id = encrypt(SECRET, RESULT_SET_SALT, id);
}

What is actually transmitted to the client is the salt and encrypted values:

{
    RESULT_SET_SALT: "randomValue1",
    387439: "Item One",
    79: "Item Two"
}

When the client selects an item to view details, the AJAX request includes the salt and encrypted value.

$.get("/ItemDetails/387439?RESULT_SET_SALT=randomValue1");

At the server the ID is decrypted using the SECRET and SALT the client included in the request.

int actualRequestedId = decrypt(387439, "randomValue1", SECRET); // result is 1

This is informative: Simple integer encryption And this: Way to encrypt a single int

Neither article talks about using a salt. Probably, if I just split the secret into two parts and transmit half of it, no one would put in the effort to crack it, but I know that type of abuse of an algorithm often breaks it and I'd prefer to do it correctly.

Any recommendations? It isn't necessary to keep the IDs as int, but I'll be transmitting large batches of them and do need to keep them small. Since there will be a large number of IDs and the encryption process will block the result UI, it shouldn't be too expensive. It would be nice if this leveraged out-of-the-box .NET (C#) encryption

EDIT: It occurs to me that another (higher bandwidth) approach is to add a random high-32-bits to each ID and encrypt it with the secret, rather than using a salt. That would work great, except since this is another abuse of an algorithm, a user's ability to generate multiple iterations from the same ID might well compromise the secret (or less importantly the individual ID).

Community
  • 1
  • 1
shannon
  • 8,664
  • 5
  • 44
  • 74
  • You should check [format preserving encryption](http://en.wikipedia.org/wiki/Format-preserving_encryption). The reason that you don't find anything about a salt is that encryption does not use salts. It may use IV's or NONCE's, but salts are generally used for key derivation functions. – Maarten Bodewes Jul 28 '12 at 11:54
  • I wonder if it would have achieved the same result to simply perform some non-secret operation on the IDs, based on a secret associated with each response (either encrypted and sent with the response and related requests-for-details, or stored in server session state). I didn't really need to encrypt the numbers. So just add or XOR them to the ID. – shannon Jul 29 '12 at 03:42
  • Encryption in this case does not seem to be the only possible answer indeed. And if it is present, it should be *part of* a well thought out scheme. Crypto is not magic (unfortunately, it would make me a magician :) ). – Maarten Bodewes Jul 29 '12 at 11:07

1 Answers1

1

Format preserving encryption is probably the thing you are looking for, but it is not easy to implement. You could use a random key that is linked to the session at the server. An attacker would be able to simply request all values, but it cannot guess them.

CTR ciphers can encrypt any value and return the same size (in bits). They have a better availability (in API's), but they require a NONCE for each encryption with the same key.

Alternatively you could simply keep a table at the server and remember a random ID to item ID. That way the attacker simply has no way to retrieve anything but the items submitted to him through the server. This might be much easier than the other solutions, but it would require additional memory for the table (of course).

This in addition to the answers submitted in the other links.

Maarten Bodewes
  • 90,524
  • 13
  • 150
  • 263
  • I like the idea of a random key in a server session key, but I also don't like it. So far, I've been able to avoid session completely (no client-linked state). I suppose encrypting/transmitting a random symmetric key for each response does the same. Another feature I was shooting for was the non-uniqueness of URLs for a unique item (detail/987343 and detail/12392 might return the same item). I realize that separating the salt out in cleartext doesn't offer this... as you said, just iterate all the encrypted IDs. Although, since most won't exist, in practice a daily quota will defeat this. – shannon Jul 29 '12 at 03:26
  • Also, it wouldn't just be more memory at the server to have a memory map of ID to random ID. It would mean either the map is persisted in my database, or not valid across servers or application restarts. I'm still processing your answer, and will probably accept it, although my architecture may be the real constraint. I suppose I could have a separate mechanism for e-mailing semi-persistent item URLs, they would just have an expiration date. – shannon Jul 29 '12 at 03:33
  • Thank you for your help, I'm accepting this answer and posting the question from another perspective. – shannon Jul 29 '12 at 03:38
  • here's my follow-up question: http://stackoverflow.com/questions/11706613/does-random-padding-an-int32-before-encryption-compromise-the-secret – shannon Jul 29 '12 at 04:16