4

I want to use Postgres, MongoDB and Neo4j together in my application. I was able to configure them all, however now each of my POJOs is backed my graphNode as well as document by aspectJ.

Is there any way to filter out which POJOs are backed by graphNodes only and which by document only?

I have a problem with saving POJOs when I do it more than twice [sic!] in a single request and I can see in the log that mongo and neo4j are both trying to create lots of instances which cause some king of deadlock.

So long story short:

  1. Is there a way to filter data mappings to configure pojo "A" to be mapped by RDBMS and graph (no document) and pojo B by document and graph (no RDBMS)
  2. Is there any sample for cross-store spring-data based application which more or less covers my problem?
  3. Why can I save two instances of pojo class in my controller one by one but when third instance is created I can notice a deadlock?

[EDIT]

What I've noticed:

  1. Mongo aspectj builder backs @Entity annotated POJOs an I don't know how to use @Entity to map POJO for Hibernate and not in MongoDB
  2. Neo4j related freeze occurs only when connected via REST and happens sometimes on 3rd sometimes on 4th and sometimes doesnt happen at all. See the controller initiation how it is done.I've tried all commented lines with no success.
  3. Transaction manager configuration must be placed in correct place, otherwise Neo4J configuration validator service fails.

[/EDIT]

I use:

  1. Spring 3.0.5
  2. SpringRoo 1.4
  3. Spring-data 1.0
  4. Postgres 9.0 + Hibernate 3.5.5
  5. Neo4j 1.3 remotely
  6. MongoDB 1.8.2 remotely
  7. All DBs are on single remote machine and works fine

[EDIT]

POM slices:

<properties>
    <roo.version>1.1.4.RELEASE</roo.version>
    <spring.version>3.0.5.RELEASE</spring.version>
    <aspectj.version>1.6.11</aspectj.version>
    <slf4j.version>1.6.1</slf4j.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <spring-security.version>3.0.5.RELEASE</spring-security.version>
    <jackson.version>1.8.0</jackson.version>
    <spring.data.mongodb.version>1.0.0.M2</spring.data.mongodb.version>
    <spring.data.graph.version>1.0.0.RELEASE</spring.data.graph.version>
    <spring.data.commons.version>1.0.0.RELEASE</spring.data.commons.version>
</properties>
    ...
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-neo4j</artifactId>
        <version>${spring.data.graph.version}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-mongodb-cross-store</artifactId>
        <version>${spring.data.mongodb.version}</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-mongodb</artifactId>
        <version>${spring.data.mongodb.version}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-neo4j-rest</artifactId>
        <version>${spring.data.graph.version}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-commons-core</artifactId>
        <version>${spring.data.commons.version}</version>
        <scope>compile</scope>
    </dependency>

  ....
  <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>aspectj-maven-plugin</artifactId>
            <version>1.0</version>
            <dependencies>
                <!-- NB: You must use Maven 2.0.9 or above or these are ignored (see 
                    MNG-2972) -->
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjrt</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjtools</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
            <executions>
                <execution>
                    <goals>
                        <goal>compile</goal>
                        <goal>test-compile</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <outxml>true</outxml>
                <aspectLibraries>
                    <aspectLibrary>
                        <groupId>org.springframework</groupId>
                        <artifactId>spring-aspects</artifactId>
                    </aspectLibrary>
                                            <aspectLibrary>
                        <groupId>org.springframework.data</groupId>
                        <artifactId>spring-data-neo4j</artifactId>
                    </aspectLibrary>
                    <aspectLibrary>
                        <groupId>org.springframework.data</groupId>
                        <artifactId>spring-data-mongodb-cross-store</artifactId>
                    </aspectLibrary>

                </aspectLibraries>
                <source>1.6</source>
                <target>1.6</target>
            </configuration>
        </plugin>

POJO:

@NodeEntity
@RooToString
@RooJavaBean
public class DElement {

@Indexed
private Long id;

@RelatedTo(direction=Direction.BOTH, elementClass=DRegion.class, type="SUBELEMENT_OF")
private Set<DElement> childElements = new HashSet<DElement>();

@Indexed(indexName = "delement-name", fulltext=true)
private String name;    



@Transactional
public void addChild(DElementchild child)
{
    this.childElements.add(child);
}

}

Controller (with loading):

    @Controller
    @RequestMapping(value="/DElements")
    public class DElementsController {

        DElementRepository DElementRepository;

        GraphDatabaseContext gdbc;

        @Autowired
        public DElementsController(DElementRepository DElementRepository, GraphDatabaseContext gdbc)
        {
            this.DElementRepository = DElementRepository;
            this.gdbc = gdbc;
            this.initElements();
        }

        @Transactional
        private void initElements()
        {
            try
            {
                DElementRepository.deleteAll();

            } 
            catch (Exception e) {} finally{}

            //Transaction txn = gdbc.beginTx();

            referenceNode.createRelationshipTo(allElements.getPersistentState(), myRelation);

            DElement naElements = new DElement().persist();
            naElements.setName("1");
            allElements.addChild(naElements);

            DElement saElements = new DElement().persist();
            saElements.setName("2");
            allElements.addChild(saElements);               

            DElement euElements = new DElement().persist();
            euElements.setName("3");
            allElements.addChild(euElements);

            DElement afElements = new DElement().persist();
            afElements.setName("4");    
            allElements.addChild(afElements);

            DElement asElements = new DElement().persist();
            asElements.setName("5");    
            allElements.addChild(asElements);

            DElement auElements = new DElement().persist();
            auElements.setName("6");
            allElements.addChild(auElements);


            //txn.success();
            //txn.finish();

        }
    }

applicationContext.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:jee="http://www.springframework.org/schema/jee"
    xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:mongo="http://www.springframework.org/schema/data/mongo"
    xmlns:graph="http://www.springframework.org/schema/data/graph"
    xsi:schemaLocation="http://www.springframework.org/schema/data/mongo http://www.springframework.org/schema/data/mongo/spring-mongo.xsd
        http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.0.xsd
        http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
        http://www.springframework.org/schema/data/graph http://www.springframework.org/schema/data/graph/datagraph-1.0.xsd
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd">
    <context:property-placeholder location="classpath*:META-INF/spring/*.properties" />
    <context:spring-configured />
    <bean id="validator"
        class="org.springframework.validation.beanvalidation.LocalValidatorFactoryBean" />

    <context:component-scan base-package="com.foobar">
        <context:exclude-filter expression=".*_Roo_.*"
            type="regex" />
        <context:exclude-filter expression="org.springframework.stereotype.Controller"
            type="annotation" />
    </context:component-scan>
    <jee:jndi-lookup id="dataSource" jndi-name="jdbc/foobar" />




    <mongo:mongo host="${foobar.mongodb.addr}" port="27017" />
    <mongo:mapping-converter base-package="com.foobar.lib.model.mongo"/>

    <bean id="mongoTemplate" class="org.springframework.data.document.mongodb.MongoTemplate">
        <constructor-arg name="mongo" ref="mongo" />
        <constructor-arg name="databaseName" value="foobar" />
        <constructor-arg name="defaultCollectionName" value="basecoll" />
    </bean>

    <bean class="org.springframework.data.document.mongodb.MongoExceptionTranslator" />

    <!-- Mongo cross-store aspect config -->
    <bean
        class="org.springframework.data.persistence.document.mongo.MongoDocumentBacking"
        factory-method="aspectOf">
        <property name="changeSetPersister" ref="mongoChangeSetPersister" />
    </bean>
    <bean id="mongoChangeSetPersister"
        class="org.springframework.data.persistence.document.mongo.MongoChangeSetPersister">
        <property name="mongoTemplate" ref="mongoTemplate" />
        <property name="entityManagerFactory" ref="entityManagerFactory" />
    </bean>


    <bean 
        id="graphDatabaseService" class="org.neo4j.kernel.EmbeddedGraphDatabase"
        destroy-method="shutdown" >
        <constructor-arg index="0" value="c:/neo4j/data/foobar" />
    </bean>
<!--  REST DOESNT WORK FOR THE MOMENT
    <bean id="graphDatabaseService" class="org.springframework.data.graph.neo4j.rest.support.RestGraphDatabase">
        <constructor-arg value="${foobar.neo4j.reststore}"/>        
    </bean> -->


<!--    <bean id="graphDatabaseContext" class="org.springframework.data.graph.neo4j.support.GraphDatabaseContext">
        <property name="graphDatabaseService" ref="graphDatabaseService"/>
    </bean> -->

    <graph:repositories base-package="com.foobar.data.repositories.neo4j" graph-database-context-ref="graphDatabaseContext"/>


    <graph:config graphDatabaseService="graphDatabaseService" entityManagerFactory="entityManagerFactory" />
    <tx:annotation-driven mode="aspectj" transaction-manager="transactionManager" />    

    <bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">     
        <property name="dataSource" ref="dataSource" />
    </bean> 
</beans>

[/EDIT]

Andrey Agibalov
  • 7,624
  • 8
  • 66
  • 111
Random
  • 4,519
  • 2
  • 38
  • 46
  • Can you provide some more details, eg. your pom and spring config and the code you use in your PoC? And perhaps some kind of lifecycle/sequence diagram that shows the actions happening in your application. – Michael Hunger Jul 18 '11 at 09:27

1 Answers1

4

Some general remarks.

You only need to pass in the entityManagerFactory to Spring Data Graph when you want to use Graph-JPA cross store persistence. An example can be found here. If you do so then you should also enable the partial=true flag on your POJOs that you want to participate in the cross store setting.

Spring Data Graph Cross store and Spring Data MongoDB cross store work differently in how they interact with JPA, SDG works from the Graph POJO side when persisting or loading entities they are (re-)connected to their JPA entity (via the @Id field).

On the other hand Spring Data MongoDB uses AspectJ to augment some of the Methods of the EntityManager to kick in lifecycle events for the document database.

Right now there is no story for integrating MongoDB and Neo4j. But as we, the project leads of both projects live in the same city and work closely together I think that should be doable.

It would be great if you could share your complete project code somewhere with us, either on github (could also be a private repo [my github id is "jexp", or per mail or dropbox). So that we could dig directly into it.

To your questions:

  • graph + rdmbs are all Entities that have partial = true and you have to have @Entity annotations on your POJO
  • I'm not sure how mongo-cross-store persistence is configured correctly
  • normally that should be configured via persistence.xml to which stores an entity is mapped?
  • i.e. perhaps we should devise an mechanism that says that configures a persistence.xml per store that wants to interact with the EntityManager/JPA/RDMBS
  • probably that also works by defining two entity managers one for the graph and the other one for mongo?
  • you should probably get it running in embedded mode of neo4j first, and look into remote server (REST) later
  • you should probably update Spring Data Graph to 1.1.M2, Neo4j to 1.4 and AspectJ to 1.6.12.M1

Please connect personally,so that we can work out all the issues and arrive at a good solution.

Michael Hunger
  • 41,339
  • 3
  • 57
  • 80
  • I cannot upgrade, see more details: http://forum.springsource.org/showthread.php?112333-spring-data-mongo-1.0.0.M3-and-graph-1.1.0.M1-issue&p=372713&posted=1#post372713 – Random Jul 22 '11 at 07:52
  • Thanks! I'm going to test it on Sunday and will share results with you. – Random Jul 22 '11 at 07:55
  • unfortunatelly i cannot use both - latest Spring Data Graph and Spring Data MongoDB because Spring Data MongoDB 1.0.0.M3 is not compatible with spring data commons 1.1.0 and is not able to find org/springframework/data/mapping/model/PersistentProperty... – Random Jul 24 '11 at 08:32
  • perhaps you can build the snapshot of Spring Data MongoDB with the updated dependency yourself? – Michael Hunger Jul 25 '11 at 22:05
  • Thanks but I'm not using Spring to spend my time on such things :) I'll wait for a stable release. It would be great if spring-data projects were tested together to make sure that all items within same release number works with each other or at least do not conflict. – Random Jul 26 '11 at 02:11