Empty list in App Engine Datastore: Java vs Python

Question

I have the following java model class in App Engine:

public class Xyz ... {
    @Persistent
    private Set<Long> uvw;
}

When saving an object Xyz with an empty set uvw in Java, I get a "null" field (as listed in the appengine datastore viewer). When I try to load the same object in Python (through remote_api), as defined by the following python model class:

class Xyz(db.Model):
    uvw = db.ListProperty(int)

I get a "BadValueError: Property uvw is required".

When saving another object of the same class in Python with an empty uvw list, the Datastore viewer prints a "missing" field.

Apparently empty lists storage handling differs between Java and Python and lead to "incompatible" objects.

Thus my question: Is there a way to, either:

• force Java to store an empty list as a "missing" field,
• force Python to gracefully accept a "null" list as an empty list when loading the object?

Or any other suggestion on how to handle empty list field in both languages.

Thanks for your answers!

Answer 1

It should work if you assign a default value to your Python property:

uvw = db.ListProperty(int, default=[])

Answer 2

I use the low-level java api, so perhaps what I am doing would be different. But before I save a collection-type data structure to the datastore, I convert it into something that the datastore naturally handles. This would include mainly Strings and ByteArrays.

It sounds like java app engine is interpreting the empty set as a null value. And python is not reading this null value correctly. You might try saving an empty set as the String value "empty set". And then have python check to see if the datastore holds that string value. If it does, it could allocate a new empty set, if not, it could read the property as a set.

Answer 3

The Java Set behavior is because Java's Collections are reference types, which default to being null.

To actually create an empty Set, declare it like this:

@Persistent
private Set<Long> uvw = new HashSet<Long>();

or using some other implementation of Set on the right side. HashSet is the most commonly used Set type, though. Other interesting set types are the two thread-safe Sets CopyOnWriteArraySet and ConcurrentSkipListSet; also the Ordered Set type LinkedHashSet and the Sorted Set type TreeSet.

Answer 4

It may work to you

uvw = db.ListProperty(int, default=[])

Its the most comment way to short it out...