10

I am trying to load all my Neo4j DB to the RAM so querying will work faster. When passing the properties map to the graph creation, I do not see the process taking more space in memory as it did before, and it is also not proportional to the space of files at disk. What could be the problem? and how can it be fixed.... Thanks

user971956
  • 3,088
  • 7
  • 30
  • 47
  • You should also make sure that you have enough memory for the memory-mapping of the files, you might want to configure that manually (http://docs.neo4j.org/chunked/milestone/configuration-io-examples.html) – Michael Hunger Apr 10 '12 at 23:15

1 Answers1

5

Neo4j loads all the data lazily, meaning it loads them into memory at first access. The caching option is just about the GC strategy, so when (or if) the references will be GCed. To load the whole graph into memory, your cache type must be strong and you need to traverse the whole graph once. You can do it like this:

// untested java code

import org.neo4j.helpers.collection.IteratorUtil;

// ...

for(Node node : graph.getAllNodes()) {
  IteratorUtil.count(node.getRelationships());
}

This way all nodes and relationships will be used once and thus loaded into the cache.

drexin
  • 24,225
  • 4
  • 67
  • 81
  • 1
    Thanks, found this also in the docs. I really wish there was just a simple flag to turn on instead all of this. – user971956 Apr 10 '12 at 14:09
  • 1
    the advantage here is that you can fine-tune it to your use-case and after all it is just 2 lines of code :) – Michael Hunger Apr 10 '12 at 23:14
  • 1
    Michael, true, but from a newbie usability perspective (which is the majority sector of a new product) having those things be dead simple is more important than the option to fine-tune. – user971956 Apr 11 '12 at 17:29
  • Can I do this without Java Code by doing a Gremlin `g.E.count()` and `g.V.count()`? – Nicholas Jul 31 '12 at 15:29
  • I don't know how gremlin works internally, but I would assume that it works. You could try it and watch your memory usage. If it grows massively, that would be a sign, that it works ;-). – drexin Jul 31 '12 at 16:15
  • @drexin could you please take a look at this question http://stackoverflow.com/questions/42148577/load-neo4j-in-memory-on-demand-for-heavy-computations? I really need your help, and pretty sure you can advice me something – VB_ Feb 10 '17 at 10:52
  • @VolodymyrBakhmatiuk Sorry, I haven't used Neo4j in years, so I don't know what changed in the meantime. – drexin Feb 10 '17 at 21:31