Neo4j - Java Heap Space Error - TraversalDescription.traverse(Node);

Question

If maxDepth is 3 code runs fine, if maxDepth = 4 this code hangs @ the call to td.traverse(start) (~line 38) & then I get:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2882) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515) at java.lang.StringBuilder.append(StringBuilder.java:189) at com.sun.jersey.core.util.ReaderWriter.readFromAsString(ReaderWriter.java:172) at com.sun.jersey.core.util.ReaderWriter.readFromAsString(ReaderWriter.java:157) at com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.readFromAsString(AbstractMessageReaderWriterProvider.java:114) at com.sun.jersey.core.impl.provider.entity.StringProvider.readFrom(StringProvider.java:73) at com.sun.jersey.core.impl.provider.entity.StringProvider.readFrom(StringProvider.java:58) at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:552) at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:505) at org.neo4j.rest.graphdb.RequestResult.extractFrom(RequestResult.java:87) at org.neo4j.rest.graphdb.ExecutingRestRequest.post(ExecutingRestRequest.java:138) at org.neo4j.rest.graphdb.ExecutingRestAPI.traverse(ExecutingRestAPI.java:486) at org.neo4j.rest.graphdb.RestAPIFacade.traverse(RestAPIFacade.java:202) at org.neo4j.rest.graphdb.traversal.RestTraversal.traverse(RestTraversal.java:162) at com.tr.cmg.alloy.dao.psd.RelationalPathSearchDAO.executeRelationalPathSearch(RelationalPathSearchDAO.java:38) at com.tr.cmg.alloy.dao.psd.RelationalPathSearchDAO.main(RelationalPathSearchDAO.java:115)

package com.tr.cmg.alloy.dao.psd;

import java.util.ArrayList;

import org.neo4j.graphdb.Direction;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.Path;
import org.neo4j.graphdb.Relationship;
import org.neo4j.graphdb.traversal.TraversalDescription;
import org.neo4j.rest.graphdb.index.RestIndex;
import org.neo4j.rest.graphdb.traversal.RestTraversalDescription;

import com.tr.cmg.alloy.aspects.audit.DaoTierAuditAspect;
import com.tr.cmg.alloy.dao.psd.relationship.RelationType;
import com.tr.cmg.alloy.dao.support.neo4j.Neo4jDaoSupport;
import com.tr.cmg.alloy.domain.psd.NodeType;
import com.tr.cmg.alloy.services.psd.rps.RelationalPathSearch;
import com.tr.cmg.alloy.services.psd.rps.RelationalPathSearchResult;

public class RelationalPathSearchDAO extends Neo4jDaoSupport{

public RelationalPathSearchResult executeRelationalPathSearch(
        RelationalPathSearch rps) {

    RestIndex<Node> personIndex = API.getIndex(NodeType.PERSON.name());

    Node start = personIndex.get("KEY", rps.getPerson1PermId()).getSingle();
    Node end = personIndex.get("KEY", rps.getPerson2PermId()).getSingle();

    Iterable<Node> nodes = null;
    Iterable<Relationship> relationships = null;

    if (start != null && end != null) {

        TraversalDescription td = createTraversalDescription(rps);
        nodes = null;
        relationships = null;
        for (Path position : td.traverse(start)) {

            // Log audit record info
            DaoTierAuditAspect
                    .storeSqlStatement("Relational Path Neo4j Query: "
                            + rps.getPerson1PermId() + " -> "
                            + rps.getPerson2PermId());
            DaoTierAuditAspect.storeSqlParameters("");

            System.out.println(position);

            if (position.endNode().equals(end)) {
                nodes = position.nodes();
                relationships = position.relationships();
                break;
            }
        }

        if(nodes == null) {
            ArrayList<Node> x = new ArrayList<Node>();
            x.add(start);
            x.add(end);
            nodes = x;
        }
    }

    return new RelationalPathSearchResult(nodes, relationships);

}

private TraversalDescription createTraversalDescription(RelationalPathSearch rps) {

    RestTraversalDescription rtd = (RestTraversalDescription) API.createTraversalDescription();
    rtd.maxDepth(4);

    return  rtd
            .depthFirst()
            .relationships(RelationType.IsDirector, Direction.BOTH)
            .relationships(RelationType.HasGraduatedFrom, Direction.BOTH)
            .relationships(RelationType.IsOfficer, Direction.BOTH)
            .relationships(RelationType.IsUnknownOfficerDirectorAt, Direction.BOTH)
            .relationships(RelationType.IsInsiderAt, Direction.BOTH)
            .relationships(RelationType.IsEquityAnalystAt, Direction.BOTH)
            .relationships(RelationType.IsMemberOrChairmanOf, Direction.BOTH)
            .relationships(RelationType.IsChiefInvestmentOfficerAt, Direction.BOTH)
            .relationships(RelationType.IsDirectorOfResearchAt, Direction.BOTH)
            .relationships(RelationType.IsPortfolioManagerAt, Direction.BOTH)
            .relationships(RelationType.IsTraderAt, Direction.BOTH)
            .relationships(RelationType.IsEconomistAt, Direction.BOTH)
            .relationships(RelationType.IsSalesProfessionalAt, Direction.BOTH)
            .relationships(RelationType.IsStrategistAt, Direction.BOTH)
            .relationships(RelationType.IsExecutiveOfficerAt, Direction.BOTH)
            .relationships(RelationType.IsShariahSupervisoryAt, Direction.BOTH)
            .relationships(RelationType.IsNonExecutiveAt, Direction.BOTH)
            .relationships(RelationType.IsVEExecutive, Direction.BOTH)
            .relationships(RelationType.IsProvidingResearchOn, Direction.BOTH)
            .relationships(RelationType.IsAnalystAt, Direction.BOTH)
            .relationships(RelationType.IsReuterMessengerUserAt, Direction.BOTH)
            .relationships(RelationType.IsProvidingAssetClassCoverageOn, Direction.BOTH)
            .relationships(RelationType.IsSpeakingLanguageOf, Direction.BOTH)
            .relationships(RelationType.IsProvidingGeographyCoverageOf, Direction.BOTH)
            .relationships(RelationType.IsUnknownSellSideBuySideAt, Direction.BOTH)
            .relationships(RelationType.IsUnknownInsiderAt, Direction.BOTH)
            .relationships(RelationType.IsUnknownResearchAnalystAt, Direction.BOTH)
            .relationships(RelationType.IsUnknownVEExecutiveAt, Direction.BOTH)
            .relationships(RelationType.IsUnknownReutersMessengerUserAt, Direction.BOTH)
            .relationships(RelationType.IsUnknownStreetEventsUserAt, Direction.BOTH);
}

public static void main(String[] args) {

    RelationalPathSearchDAO dao = new RelationalPathSearchDAO();

    RelationalPathSearch rps = new RelationalPathSearch();
    rps.setPerson1PermId("34414591164");
    rps.setPerson2PermId("34414710307");

    RelationalPathSearchResult result = dao.executeRelationalPathSearch(rps);
    result.toJSON();

}

}

score 1 · Answer 1 · edited May 23 '17 at 11:48

1

You need to increase the JVM maximum heap memory size.

What is your current JVM memory settings?

edited May 23 '17 at 11:48

Community

1
1

answered Jan 08 '13 at 20:00

Werner Kvalem Vesterås

10,226
5
43
50

I've set the Max Memory to -Xmx1024m & I still get the error. The DB only has 1.6M nodes, ... so this really shouldn't be a problem .. – Black Dog Jan 08 '13 at 21:01

score 0 · Answer 2 · edited May 23 '17 at 11:43

0

You can increase memory for heap space using "-XMX" parameter. You can refer this also...How to increase heap space in memory for java

edited May 23 '17 at 11:43

Community

1
1

answered Jan 09 '13 at 09:24

Sushil Jain

477
1
5
13

score 0 · Answer 3 · answered Jan 09 '13 at 14:33

It looks like you're trying to find paths between two nodes, right? If you use allPaths or allSimplePaths algorithm through the REST API instead it sould be faster and require less memory as it would be a bi-directional traversal.

Not sure if the RestGraphDatabase supports it though. Either way you could have a look at http://docs.neo4j.org/chunked/stable/rest-api-graph-algos.html

score 0 · Answer 4 · answered Jan 09 '13 at 14:35

0

It is solved, the approach taken pulls the whole database into json and across the wire to do something that should be run inside the db.

It is solved in this GitHub issue.

answered Jan 09 '13 at 14:35

Michael Hunger

41,339
3
57
80

Neo4j - Java Heap Space Error - TraversalDescription.traverse(Node);

4 Answers4