Creating unique app engine datastore entity by numeric ID without conflicts - python

Question

I'm trying to create a numeric id for an entity without conflicts when multiple people could potentially be trying to create entities at once (the id will be seen and used by the apps users, so ideally it will be under 6 digits in length - auto assigned ID's probably aren't acceptable).

My current solution seems overly cumbersome and possibly doesn't even work to solve conflicts.

I have an ID counter that I'm storing within a different App entity:

class App(ndb.Model):
    # counter to store number of projects
    num_projects     = ndb.IntegerProperty(default = 0)

And the actual projects:

class Project(ndb.Model):
    date_created  = ndb.DateTimeProperty(auto_now_add = True)
    last_modified = ndb.DateTimeProperty(auto_now = True)

    owner       = ndb.KeyProperty(required = True)
    owner_name  = ndb.StringProperty(required = True)

    name        = ndb.StringProperty(required = True)

and a handler class for creating projects: (that could be called simultaneously)

def make_new_project(self):
    pdt = App.by_id('pdt')
    pdt.num_projects = pdt.num_projects + 1
    if not Project.by_id(pdt.num_projects):
        project = Project(id=pdt.num_projects)
        project.put()
        pdt.put()
        return project
    else:
        return self.make_new_project()

The intention of the handler is to check if the project ID is already being used (someone literally put a new project after I retrieved the App counter) and if not, create the project with the new counter number, then store both to the DB.

I feel this is still kinda buggy and could lead to race conditions?? Is there a better way? Should I be pre-assigning keys and if so how do I do that? Do I store the assigned keys into my App model for later retrieval?

I get easily confused about the run-time of certain functions. If I add a function to assign keys to my main.py, does this run every time a new instance is started?

To clarify for some of the questions asked below, the projects should be findable by nuerics as the users of the existing system prefer to quote a project number when talking to other staff... ie: "I'm phoning regarding project 13670" and they can tell which project is older by sequential numbering, otherwise I would just generate a random 5 digit number and check it's not already in use before assigning.

Any help, clarification, etc... would be greatly appreciated.

Have you looked into transactions or using `get_or_insert`? Also, you can use a numeric id instead of a string id so you can leave out the `str`. — new name, Jan 15 '16 at 15:11
This doesn't seem like a good design overall... If you explain why you want consecutive ids, people could probably suggest a better design. — new name, Jan 15 '16 at 15:14
Thanks for the comments. Feedback from the staff that will be using this system is that to align with previous systems and make finding a "project" very easy they tend to just ask others for the ID number of the project... IE: "I'm calling about project 7045" when talking with others. Ideally the project numbers should sequential so that you can tell the age of a project by its ID number alone. 7044 was the project before 7045. — Mike Fitzbaxter, Jan 18 '16 at 04:17

score 1 · Answer 1 · edited May 23 '17 at 11:44

1

Use the NDB-generated key: it is designed for multiple concurrency and does it well.

Then convert the generated key to something more friendly: base36 should do the trick.

eg:

9999999999999999 (16 length)

becomes

2QGPCKVNG1R (11 length)

Specifically:

friendlyValue = base36encode(entity.key.id())

edited May 23 '17 at 11:44

Community

1
1

answered Jan 15 '16 at 03:14

ChrisC73

1,833
14
14

As in my comment above, whilst this shortens the ID field from 16 -> 11 in length, it doesn't make it easier to read out or remember. Having sequential IDs from 00001 -> 99999 makes them easy to recite over the phone or in emails. – Mike Fitzbaxter Jan 18 '16 at 04:18
@Mike Baxter : Understood. You may wish to look for a python gae [mutex solution](http://xion.org.pl/2011/12/10/synchronization-through-memcache/) to adapt for your needs. As this one relies on memcache there is the possibility of inconsistency. – ChrisC73 Jan 18 '16 at 23:17

Creating unique app engine datastore entity by numeric ID without conflicts - python

1 Answers1