ndb many to many, retrieve list of one of the relation

Question

I've this table

class ClubMembership(GCModel):
    member = ndb.KeyProperty(kind='User', required=True)
    club = ndb.KeyProperty(kind='Club', required=True)
    is_active = ndb.BooleanProperty(default=True)
    membership_type = ndb.StringProperty(choices=set(["MEMBER", "TRAINER", "OWNER"]), default="MEMBER",
                                     required=True)

Then in the Table Club i've this

class Club(GCModel):
    @property
    def members(self):
        return ClubMembership.query(ndb.AND(ClubMembership.club == self.key,
                                            ClubMembership.membership_type == "MEMBER",
                                            ClubMembership.is_active == True))

what i actually need is to retrive the list of User. how should i do?

fetch this query, than have a for loop that fetches each member using its key?

i tried to use ndb.get_multi using the projection on member but it does not work. something like this

members= ClubMembership.query(ndb.AND(ClubMembership.club == self.key,
                                            ClubMembership.membership_type == "MEMBER",
                                            ClubMembership.is_active == True),projection=['member']).fetch()
ndb.get_multi(members)

score 3 · Accepted Answer · answered Jan 08 '15 at 10:09

3

get_multi works on a list of keys, you have a list of ClubMembership instances. You need to get the member key for each one:

memberships = ClubMembership.query(...)
member_keys = [m.member for m in memberships]
members = ndb.get_multi(member_keys)

answered Jan 08 '15 at 10:09

Daniel Roseman

588,541
66
880
895

That was my feeling. is there any performance difference if i do a for loop (because i need some other processing) `for membership in memberships: user = membership.member.get()`? – EsseTi Jan 08 '15 at 10:18
See [the docs](https://cloud.google.com/appengine/docs/python/ndb/entities#multiple) on that - doing it in the loop will cause a separate RPC for each entity, which will definitely be slower than a single get_multi call. Better to get them all at once and do your processing on the results. – Daniel Roseman Jan 08 '15 at 10:21
I see, so, what should be the best way to get all the user and their type of membership? because the type of membership is inside the membership list. Thus i get the users with the get_multi, but now: how can i match the user with the one in the membership list to retrieve its membership type? will the two list be ordered the same? – EsseTi Jan 08 '15 at 10:40
@EsseTi easy. use a loop over retrieved users, and one more nested loop on each membership looking for member matches. – Dmytro Sadovnychyi Jan 08 '15 at 11:36
1

@EsseTi if you want to get all members AND their type of memebership, you can do something like zip(memberships, ndb.get_multi([m.member for m in memberships])) – marianosimone Jan 08 '15 at 18:13
@Esseti I added the "bonus" to the actual answer. Another trick, if you DON'T need the Membership data, is to fetch(keys_only=True) the memberships, and extract the User keys from their (in case you are still using the approach we discussed here: http://stackoverflow.com/questions/26740505/efficient-way-to-store-relation-values-in-ndb/26746542) – marianosimone Jan 09 '15 at 16:05
you mean somenthing like this? `User.get_by_key(membership.key.split("|")[1])` – EsseTi Jan 09 '15 at 16:59

score 1 · Answer 2 · answered Jan 08 '15 at 10:33

1

You should also consider to store membership information inside of user's entity. This way you can get all users in a single query, which is much faster.

class Membership(ndb.Model):
  club = ndb.KeyProperty(kind='Club', required=True)
  is_active = ndb.BooleanProperty(default=True)
  membership_type = ndb.StringProperty(
    choices=set(["MEMBER", "TRAINER", "OWNER"]), default="MEMBER", required=True)

class User(ndb.Model):
  memberships = ndb.StructuredProperty(Membership, repeated=True)

class Club(ndb.Model):
  @property
  def members(self):
    membership = Membership(
      User.memberships.club == self.key,
      User.memberships.membership_type == 'MEMBER',
      User.memberships.is_active == True)
    return User.query(User.memberships == membership)

answered Jan 08 '15 at 10:33

Dmytro Sadovnychyi

6,171
5
33
60

I see, the fact is that i'm quite skeptical about the repeated entity beacuse of this http://stackoverflow.com/questions/15377119/gae-ndb-design-performance-and-use-of-repeated-properties/15418435#15418435 – EsseTi Jan 08 '15 at 10:40
1

It looks like really rare case when the user will be a member of many clubs, but its your decision. – Dmytro Sadovnychyi Jan 08 '15 at 11:07
but i've the same problem with courses, and people may subscribe to many courses.. – EsseTi Jan 08 '15 at 11:20
anyway de-normalisation of data -- its a good practice for app engine datastore, you should be able to generate the whole page with a single query, to make it blazing fast. I think that it should work even for courses. – Dmytro Sadovnychyi Jan 08 '15 at 11:22
You are right, one thing. To get all the clubs a user is member of, what i should do? iterate the `memberships` of the user and then checking, one by one, if the membership is of type `MEMBER`? – EsseTi Jan 08 '15 at 12:33

ndb many to many, retrieve list of one of the relation

2 Answers2

Linked