1

I am quite new to Neo4j and using the Neo4j desktop version with the Javascript driver. My graph has about 30.000 nodes and 40.000 edges. The final approach would be to get all simple cycles in the graph but I am facing a heap overflow while trying to execute the following cypher query:

let res = await session.run('MATCH p=(n)-[*2..4]-(n) RETURN nodes(p)')

This is of course because the query tries to return all the simple cycles found which is way to much for my heap and unhappily increasing the memory for node.js isn't an option.

So is there any way I could bypass this problem and get all simple cycles from the graph?

Here is the heap overflow error I got:

/Users/paulus/.nvm/versions/node/v15.10.0/bin/node /Users/paulus/routeplanner/RouteFinder.js

<--- Last few GCs --->

[39630:0x104e00000]   227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 


<--- JS stacktrace --->


<--- Last few GCs --->

[39630:0x104e00000]   227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory

<--- Last few GCs --->

[39630:0x104e00000]   227813 ms: Scavenge (reduce) 2033.5 (2077.9) -> 2032.1 (2072.1) MB, 113.5 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228171 ms: Scavenge (reduce) 2036.5 (2072.6) -> 2035.5 (2075.6) MB, 28.6 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 
[39630:0x104e00000]   228558 ms: Scavenge (reduce) 2039.1 (2075.6) -> 2038.5 (2080.4) MB, 28.2 / 0.0 ms  (average mu = 0.577, current mu = 0.321) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory

Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
Paulus
  • 162
  • 7

1 Answers1

1

It's difficult to give you an exact solution without knowing the data model of your graph, but it may help to use skip/limit to get data in smaller bulks.

This get the first 100 patterns:

MATCH p=(n)-[*2..4]-(n) RETURN nodes(p) SKIP 0 LIMIT 100

And this the next 100:

MATCH p=(n)-[*2..4]-(n) RETURN nodes(p) SKIP 100 LIMIT 100
Dharman
  • 30,962
  • 25
  • 85
  • 135
dgrana
  • 108
  • 6
  • This works perfect but how do I find out the limit? – Paulus Apr 28 '21 at 11:58
  • If you refer to the total of patterns you can first make a query with a count (MATCH p=(n)-[*2..4]-(n) RETURN count(p)) to know the total. But if you refer to the bulk size, you have to calculate it based on the amount of data you have and its size (big enough not to do a lot of queries but not so big that it can fail). – dgrana Apr 28 '21 at 12:07
  • I was looking for the total amount of patterns... thanks! – Paulus Apr 28 '21 at 12:27