Effectively clear Neo4j database

Question

This is my previous question Clear Neo4j Embedded database

Right now I understand that I don't need to shutdown database, I only need to wipe all data inside of this database.

I use the following method:

public static void cleanDb(Neo4jTemplate template) {
    template.query("MATCH (n) OPTIONAL MATCH (n)-[r]-() DELETE n,r", null);
}

but it's not work properly on a large data sets.

Also, with a new version of Spring Data Neo4j I can't use Neo4jHelper.cleanDb(db);

Is any ways to correctly and effectively clean the database state without database shutdown/deleting ?

UPDATED

I have implemented following util class with cleanDb method

public class Neo4jUtils {

    final static Logger logger = LoggerFactory.getLogger(Neo4jUtils.class);

    private static final int BATCH_SIZE = 10;

    public static void cleanDb(Neo4jTemplate template) {
        logger.info("Cleaning database");

        long count = 0;
        do {
            GraphDatabaseService graphDatabaseService = template.getGraphDatabaseService();
            Transaction tx = graphDatabaseService.beginTx();
            try {
                Result<Map<String, Object>> result = template.query("MATCH (n) WITH n LIMIT " + BATCH_SIZE + " OPTIONAL MATCH (n)-[r]-() DELETE n, r RETURN count(n) as count", null);
                count = (long) result.single().get("count");
                tx.success();
                logger.info("count: " + count);
            } catch (Throwable th) {
                logger.error("Error while deleting database", th);
                throw th;
            } finally {
                tx.close();
            }
        } while (count > 0);

    }

}

right now it hangs on the line:

tx.close();

How to fix it, what am I doing wrong ?

Also, after number of experiments I have noticed that I can clean the database as many times as I want only on the working application. Right after application restart(I kill application process from console) cleanDb method stops working on this existing database and hangs.

No issues in the messages.log, everything looks fine:

2015-07-25 23:06:59.285+0000 INFO  [o.n.k.EmbeddedGraphDatabase]: Database is now ready

I have no idea what can be wrong.. please help to solve this issue.

I use:

neo4j version 2.2.3
lucene version 3.6.2
spring-data-neo4j version 3.4.0.M1

IMPORTANT UPDATE

I noticed that everything work properly if I use graphDatabaseService.shutdown(); method before terminating of my application.. Otherwise database is destroyed(Neo4j server also hangs on this corrupted db).

Is any way to make Neo4j Embedded database more fault tolerant ? I will lose all my data after the first error (for example blackout event) in the production environment..

score 0 · Answer 1 · answered Jul 25 '15 at 12:43

0

I don't know how it works with Spring Data, but in general you should try to delete nodes/relationships in batches.

Cypher query:

MATCH (n)
WITH n LIMIT 10000
OPTIONAL MATCH (n)-[r]-()
DELETE n, r
RETURN count(n)

In your application you do:

while return_value > 0:
    run_delete_query()

Depending on your memory you can of course increase the LIMIT.

answered Jul 25 '15 at 12:43

Martin Preusse

9,151
12
48
80

Thanks for you answer. The application hangs on this query also - Result> result = template.query("MATCH (n) WITH n LIMIT 1000 OPTIONAL MATCH (n)-[r]-() DELETE n, r RETURN count(n)", null); As you can see, I have reduced LIMIT even to 1000 – alexanoid Jul 25 '15 at 15:42
OK, try 5 maybe to see if it's still a memory issue. – Martin Preusse Jul 25 '15 at 17:50
OK, does it work with a Cypher query in the neo4j web interface? – Martin Preusse Jul 25 '15 at 17:52
It works in Neo4j Web Interface - Deleted 344205 nodes, deleted 988809 relationships, statement executed in 45721 ms. – alexanoid Jul 25 '15 at 18:45
This query(with LIMIT 1000) "MATCH (n) WITH n LIMIT 1000 OPTIONAL MATCH (n)-[r]-() DELETE n, r RETURN count(n)" works on -Xmx512m in Neo4j Web Interface ... The full delete doesn't work even with -Xmx4g – alexanoid Jul 25 '15 at 19:06
I have updated my question, right now it hangs on tx.close() – alexanoid Jul 25 '15 at 19:27
I noticed that everything work properly if I use graphDatabaseService.shutdown(); method before terminating of my application.. Otherwise database is destroyed(Neo4j server also hangs on this corrupted db) – alexanoid Jul 26 '15 at 09:36

Effectively clear Neo4j database

1 Answers1

Linked