Gaelyk: How to perform datastore queries on collection attributes

Question

The Gaelyk tutorial provides some nice low level wrappers to the datastore and this Gaelyk google groups article describes an easy way to model relationships by simply storing the keys in a collection on the entity.

My question is how can I perform queries on the values within the collections? Here is some example code to clarify...

def b1 = new Entity("Book")
b1.title = "Book1"
b1.save()

def b2 = new Entity("Book")
b2.title = "Book2"
b2.save()

def author1 = new Entity("Author")  
author1.books = [b1.key, b2.key]
author1.name = "Chris"
author1.save()

// It is easy to simply query the Author entity for a standard string property
def query = new Query("Author")
query.addFilter("name", Query.FilterOperator.EQUAL, "Chris")
PreparedQuery preparedQuery = datastore.prepare(query)
def authors = preparedQuery.asList(withLimit(1))
assert authors[0].name == "Chris"

// But I can't find out how to query a collection property eg books on the Author entity
def query2 = new Query("Author")
// I would like to do something to return the entity if the value is in the collection property
// eg if there could be something like 'CONTAINS' criteria ...
// Unfortunately Query.FilterOperator.CONTAINS doesn't exist...
query2.addFilter("books", Query.FilterOperator.CONTAINS, b2.key)
PreparedQuery preparedQuery2 = datastore.prepare(query2)
def authors2 = preparedQuery2.asList(withLimit(1))
assert authors2[0].name == "Chris"

How can I create a query that that searches for matches within a collection property of an Entity? ie how to recreate the functionality of the mythical 'FilterOperator.CONTAINS' query above?

score 6 · Accepted Answer · answered Aug 02 '11 at 18:14

Answering just for the sake of users landing to this page in future:

Query.FilterOperator.EQUAL will find inside the list of keys as well (it works as CONTAINS, in case of lists). So now your second case looks like:

def query2 = new Query("Author")
query2.addFilter("books", Query.FilterOperator.EQUAL, b2.key)
PreparedQuery preparedQuery2 = datastore.prepare(query2)
def authors2 = preparedQuery2.asList(withLimit(1))
assert authors2[0].name == "Chris"

and the assertion passes :)

This may look strange at first sight but it is indeed a great feature of DataStore.

score 3 · Answer 2 · edited Feb 09 '11 at 21:09

3

One way to do this would be to have the author key as one of the fields in your entity "Book". Therefore, you can query the book first and then get the authors for it.

def author1 = new Entity("Author")  
author1.name = "Chris"
author1.save()

def b1 = new Entity("Book")
b1.title = "Book1"
b1.authors = [author1.key]
b1.save()

author1.books = [ b1.key ]
author1.save()

def book = datastore.get(b1.key)    
def chrisAuthors = book.authors.findAll { authorKey -> datastore.get(authorKey).name == 'Chris' }
assert chrisAuthors.size() == 1

edited Feb 09 '11 at 21:09

Chris

3,552
2
26
36

answered Feb 05 '11 at 20:09

Benjamin Muschko

32,442
9
61
82

thanks, I was hoping it was possible to avoid having to constantly keep track of and make sure the back references are in sync. The Python GAE has the [ReferencePropery](http://code.google.com/appengine/articles/modeling.html) which automatically keeps the back links consistent. Do you know if there is a Java/Groovy equivalent to the ReferenceProperty? – Chris Feb 09 '11 at 21:06
@Chris: If you really want to model the relationships I guess you have to go with GAE's JDO/JPA implementation. However, I do not recommend using them. They can't fully abstract the fact that BigTable is not a relational database. Furthermore, they seem to be buggy/hacky when it comes to more complex relations. It also introduces new JAR dependencies which will increase your application's startup time. [Objectify](http://code.google.com/p/objectify-appengine/) might be a good way to go instead. – Benjamin Muschko Mar 19 '11 at 14:10

score 2 · Answer 3 · answered Aug 31 '11 at 07:23

By the way, a little update on this topic: you might want to have a look at the new Query DSL feature of Gaelyk 1.0, which was released recently.

http://gaelyk.appspot.com/tutorial/app-engine-shortcuts#query

This should simplify a lot of the queries one can make against the datastore.

Gaelyk: How to perform datastore queries on collection attributes

3 Answers3