22

What is the maximum number of characters that can be used to define the key_name of a datastore entity?

Is it bad to have very long key_names?

For example: Lets say we use key_names of a 170 characters, which is the length of a Twitter message 140 plus 10 numeric characters for latitude and 10 for longtitude and 10 for a timestamp.

(Reasoning of such a key_name: So by using such a key_name we can easily and quickly be sure of no duplicate postings, since the same message should not come from the same place and time more than once.)

Dan McGrath
  • 41,220
  • 11
  • 99
  • 130
b_dev
  • 2,568
  • 6
  • 34
  • 43
  • I can add another reason for using long key names. One of my datatypes represent a relationship between two other entities (of any type), and the key name of my entities are the two encoded keys with an underscore in the middle. If the entities have parents (and grandparents etc), the keys can get pretty long. – yngling Apr 08 '14 at 15:58

2 Answers2

43

key names are limited to 500 characters, just like string property values. see e.g. Key.to_path(), which calls ValidateString():

http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/datastore_types.py#413

which defaults max_len to _MAX_STRING_LENGTH, which is 500:

http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/datastore_types.py#87

ryan
  • 2,687
  • 1
  • 29
  • 38
  • Is this true for ndb? What about java and go? I suspect Nick's answer is correct wrt the backend - it's just the software that is imposing this limit. – mjibson Mar 17 '13 at 00:22
  • iirc the backend enforces the same limit. (i wrote much of it, but that was a while ago, and i don't have access to the code any more. :P) in any case, it's easy to check in ndb, go, etc. – ryan Mar 17 '13 at 06:50
  • 2
    Just to confirm in the present, what @ryan said is still true of the validation in the backend. – Patrick Costello Aug 04 '14 at 18:23
  • @PatrickCostello I have a couple more in-depth questions to the key-name limit and decided to post them as a new question. If you might have any insights on these, I would love to ask you to take a quick look here: http://stackoverflow.com/questions/33044281/understanding-maximum-length-of-an-appengine-key-name-in-the-java-api Thank you. – Markus A. Oct 09 '15 at 17:47
12

There's no hard maximum - the maximum length of a key name is the maximum length of a key, less some overhead, and keys can get pretty long.

It is bad to have very long key names, however: Apart from storing and retrieving it, every index entry contains the key name it's referring to, so longer key names mean higher indexing overhead. If you want to ensure uniqueness over a large text, your best option is to make the key name the MD5 or SHA1 sum of the input, which ensures both uniqueness and a short(-ish) key name.

Nick Johnson
  • 100,655
  • 16
  • 128
  • 198
  • 2
    Well, generally no hashing -- nor MD5, nor SHA1 can ensure uniqueness. Practically speaking -- Sun will blow up sooner than you get collision of these hashes :) – Dzmitry Lazerka Sep 21 '14 at 22:36
  • https://arstechnica.com/information-technology/2020/01/pgp-keys-software-security-and-much-more-threatened-by-new-sha1-exploit/ – ryan Jan 13 '20 at 20:50