GAE: Exception while allocating 16 digit ids

Question

I notice problem with allocating IDs on google app engine while using datastore. In my application I have a set of data that have to be initially uploaded. Data has been prepared on test appengine environment so it has autogenerated values for ID fields. Since I want to preserve these values I'm recreating entities by using remote API with Objectify as a separate process. After upload I want to make sure that used IDs will be removed from value range for autogenerator. I'm using DatastoreService.allocateIdRange with range of single long value. Everything works fine on dev server but on appspot for some values (16 digits values) I receive "Exceeded maximum allocated IDs" IllegalArgumentException.

Is there any limitation of allocateIdRange call (I have found none in documentation)?

Below is a sample code I'm using for id allocation for datastore after upload:

DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
String kind = Key.getKind(clazz);

PreparedQuery query = datastore.prepare(new Query(kind).setKeysOnly());
KeyRange keyRange = null;
Long id = null;

for (Entity entity : query.asIterable()) {
    id = (Long) entity.getKey().getId();
    keyRange = new KeyRange(null, kind, id, id);
    DatastoreService.KeyRangeState state = datastore.allocateIdRange(keyRange);
}

score 3 · Answer 1 · answered Apr 18 '14 at 20:30

3

This is a known issue with allocateIdRange(). A better error message would be "You can't call allocateIdRange() on scattered ids".

Scattered ids are the default since 1.8.1 and have values >= 2^52. Unfortunately we don't currently expose an API to reserve these ids.

answered Apr 18 '14 at 20:30

Rohwer

149
1

Google's incomplete documentation can be infuriating!!! I'm doing a significant rewrite of my app based on `allocate_id_range` and I tested it on an older, smaller id and it worked fine. Now I find it doesn't work for the newer, larger id's, and I've wasted a huge amount of time. – new name Nov 28 '14 at 15:43

score 1 · Answer 2 · edited May 23 '17 at 10:29

1

It sounds like you may be trying to allocate an ID larger than the max allowed ID. This is limited by the largest integer size in javascript, which is 2^53.

Here is the page describing the App Engine limitation and the largest javascript int.

edited May 23 '17 at 10:29

Community

1
1

answered Apr 01 '14 at 22:11

Patrick Costello

3,616
17
22

Thanks for your answer, but I'm using ids that were previously generated by GAE - and are less than 2^53. – user3482048 Apr 07 '14 at 08:47
Hmm. I would recommend filing a production ticket on the App Engine site so that you can provide your app id and someone can look more in-depth at your app: https://code.google.com/p/googleappengine/issues/entry?template=Production%20issue – Patrick Costello Apr 07 '14 at 16:38

GAE: Exception while allocating 16 digit ids

2 Answers2