Neo4j Spring data POC for social RESTful layer

Question

Starting to work on a new project... RESTful layer providing services for social network platform.
Neo4j was my obvious choice for main data store, I had the chance to work with Neo before but without exploiting Spring Data abilities to map POJO to node which seems very convenient.

Goals:

1. The layer should provide support resemble to Facebook Graph API, which defines for each entity/object related properties & connections which can be refer from the URL.
   FB Graph API

2. If possible I want to avoid transfer objects which will be serialized to/from domain entities and use my domain pojo's as the JSON's transferred to/from the client.

Examples:

• HTTP GET /profile/{id}/?fields=...&connections=... the response will be Profile object contains the requested in the URL.

• HTTP GET /profile/{id}/stories/?fields=..&connections=...&page=..&sort=... the response will be list of Story objects according to the requested.
  

Relevant Versions:

• Spring Framework 3.1.2
• Spring Data Neo4j 2.1.0.RC3
• Spring Data Mongodb 1.1.0.RC1
• AspectJ 1.6.12
• Jackson 1.8.5

To make it simple we have Profile,Story nodes and Role relationship between them.

public abstract class GraphEntity {
@GraphId
protected Long id;
}

Profile Node

@NodeEntity
@Configurable
public class Profile extends GraphEntity {

// Profile fields
private String firstName;
private String lastName;

// Profile connections  
@RelatedTo(type = "FOLLOW", direction = Direction.OUTGOING)
private Set<Profile> followThem;

@RelatedTo(type = "BOOKMARK", direction = Direction.OUTGOING)
private Set<Story> bookmarks;

@Query("START profile=node({self}) match profile-[r:ROLE]->story where r.role = FOUNDER and story.status = PUBLIC")
private Iterable<Story> published;
}

Story Node

@NodeEntity
@Configurable
public class Story extends GraphEntity {

// Story fields
private String title;
private StoryStatusEnum status = StoryStatusEnum.PRIVATE;

// Story connections
@RelatedToVia(type = "ROLE", elementClass = Role.class, direction = Direction.INCOMING)
private Set<Role> roles;
}

Role Relationship

@RelationshipEntity(type = "ROLE")
public class Role extends GraphEntity {

@StartNode
private Profile profile;
@EndNode
private Story story;

private StoryRoleEnum role;
}

At first I didn't use AspectJ support, but I find it very useful for my use-case cause it is generating a divider between the POJO to the actual node therefore I can request easily properties/connections according to the requests and the Domain Driven Design Approach seems very nice.

Question 1 - AspectJ:

Let's say I want to define default fields for an object, these fields will be returned to the client whether if requested in the URL or not...so I have tried @FETCH annotation on these fields but it seems it is not working when using AspectJ. At the moment I do it that way..

public Profile(Node n) {
    setPersistentState(n);
    this.id = getId();
    this.firstName = getFirstName();
    this.lastName = getLastName();  
}

Is it the right approach to achieve that? does the @FETCH annotation should be supported even when using AspectJ?
I will be happy to get examples/blogs talking about AspectJ + Neo4j didn't find almost anything....

Question 2 - Pagination:

I would like to support pagination when requesting for specific connection for example

/profile/{id}/stories/ , if stories related as below

// inside profile node
@RelatedTo(type = "BOOKMARK", direction = Direction.OUTGOING)
private Set<Story> bookmarks; 

/profile/{id}/stories/ ,if stories related as below

 // inside profile node
@Query("START profile=node({self}) match profile-[r:ROLE]->story where r.role = FOUNDER and story.status = PUBLIC")
private Iterable<Story> published;

Is pagination is supported out of the box with either @Query || @RelatedTo || @RelatedToVia using Pageable interface to retrieve Page instead of Set/List/Iterable? the limit and the sorting should be dynamic depending on the request from the client... I can achieve that using Cypher Query DSL but prefer to use the basic.. other approaches will be accepted happily.

Question 3 - @Query with {self}:

Kind of silly question but I can't help it :), it seems that when using @Query inside the node entity ( using {self} parameter } the return type must be Iterable which make sense.. lets take the example of...

// inside profile node
@Query("START profile=node({self}) match profile-[r:ROLE]->story where r.role = FOUNDER and story.status = PUBLIC")
private Iterable<Story> published;

When published connection is requested:

    // retrieving the context profile
    Profile profile = profileRepo.findOne(id);
    // getting the publishe stories using AspectJ - will redirect to the backed node
    Iterable<Story> published = profile.getPublished();
    // set the result into the domain object - will throw exception of read only because the type is Iterable
    profile.setPublished(published);

Is there a workaround for that? which is not creating another property which will be @Transiant inside Profile..

Question 4 - Recursive relations:

I am having some problems with transitive / recursive relations, when assigning new Profile Role in Story the relation entity role contain @EndNode story , which contain roles connection...and one of them is the context role above and it is never end :)... Is there a way to configure the spring data engine not to create these never ending relations?

Question 5 - Transactions:

Maybe I should have mentioned it before but I am using the REST server for the Neo4j DB, from previous reading I understand that there is not support out-of-the-box in transactions? like when using the Embedded server I have the following code...

    Profile newProfile = new Profile();
    newProfile.getFollowThem().add(otherProfile);
    newProfile.getBookmarks().add(otherStory);
    newProfile.persist(); // or profileRepo.save(newProfile)

will this run in transaction when using REST server? there are few operations here, if one fail all fail?

Question 6 - Mongo + Neo4j:

I need to store data which don't have relational nature.. like Feeds, Comments , Massages.
I thought about an integration with MongoDB to store these.. can I split domain pojo fields/connections to both mongo/neo4j with cross-store support? will it support AspectJ?

That is it for now.... any comments regarding any approach I presented above will be welcome.. thank you.

Answer 1

Starting to answer, by no means complete:

Perhaps upgrade to the the .RELEASE versions?

Question 1

If you want to serialize AspectJ entities to JSON you have to exclude the internal fields generated by the advanced mapping (see this forum discussion).

When you use the Advanced Mapping @Fetch is not necessary as the data is read-through from the database anyway.

Question 2

For the pagination for fields, you can try to use a cypher-query with @Query and LIMIT 100 SKIP 10 as a fixed parameter. Otherwise you could employ a repository/template to actually fill a Collection in a field of your entity with the paged information.

Question 3

I don't think that the return-type of an @Query has to be an Iterable it should also work with other types (Collections or concrete types). What is the issue you run into?

For creating recursive relationships - try to store the relationship-objects themselves first and only then the node-entities. Or use template.createRelationshipBetween(start, end, type, allowDuplicates) for creating the relationships.

Question 5

As you are using SDN over REST it might not perform very well, as right now the underlying implementation uses the RestGraphDatabase for fine-grained operations and the advanced mapping uses very fine grained calls. Is there any reason why you don't want to use the embedded mode? Against a REST server I would most certainly use the simple-mapping and try to handle read operations mostly with cypher.

With the REST APi there is only one tx per http-call the only option of having larger transactions is to use the rest-batch-api.

There is a pseudo-transaction support in the underlying rest-graph-database which batches calls issued within a "transaction" to be executed in one batch-rest-request. But those calls must not rely on read-results during the tx, those will only be populated after the tx has finished. There were also some issues using this approach with SDN so I disabled it for that (it is a config-option/system-property for the rest-graphdb).

Question 6

Right now cross-store support for both MongoDB and Neo4j is just used against a JPA / relational store. We discussed having cross-store references between the spring-data projects once but didn't follow up on this.